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INTRODUCTION 


BACKGROUND 

The  Naval  Ocean  Systems  Center,  under  the  sponsorship  of  the  Naval  Medical 
Research  and  Development  Command,  is  performing  development  work  on  a medical  data 
transfer  network  called  the  Remote  Medical  Diagnosis  System  (RMDS).  This  system  will 
link  remote  sites  (ships  or  stations)  lacking  professional  medical  staffs  with  diagnostic 
centers  having  full  medical  capability. 

Since  the  RMDS  will  utilize  existing  Navy  communication  systems,  it  is  necessary  to 
stay  within  the  bandwidth  constraints  imposed  by  these  systems.  For  this  reason  only  slow- 
scan  television  (SSTV)  is  considered  feasible  for  use  by  the  RMDS  to  transmit  video  images. 
SSTV  transmits  a static  image  (ie,  does  not  show  motion)  and  typically  may  take  from  4 
seconds  to  10  minutes  to  transmit  a single  image.  The  time  required  to  transmit  an  image  is 
dependent  upon  the  transmission  mode  (analog  or  digital),  the  bandwidth  (analog)  or  data 
rate  (digital),  and  the  level  of  resolution  and  gray  scale  to  be  achieved.  Utilizing  a combina- 
tion of  existing  Navy  communication  systems  and  standard  telephone  lines,  analog  trans- 
missions are  limited  to  a bandwidth  of  3 kHz  and  digital  transmissions  are  currently  limited 
to  2400  bits  per  second  ( baud).  Using  a 2400-baud  digital  link,  a system  capable  of  5 1 2 
television  lines,  256  picture  elements  per  line,  and  6 bits  per  picture  element  requires  6 min- 
utes and  6 seconds  to  transmit  a single  image.  Operational  demands  of  Navy  communication 
systems  make  it  desirable  to  minimize  the  amount  of  data  to  be  transmitted  in  order  to 
constrain  transmissions  to  a reasonable  amount  of  time,  and  still  provide  sufficient  resolution 
to  allow  accurate  diagnoses. 

Because  of  these  technical  and  operational  factors,  the  resolution  and  gray  scale  to  be 
provided  for  transferring  and  displaying  radiographic  data  should  be  no  more  than  is  needed 
to  meet  essential  requirements.  The  gray  scale  requirements  for  display  have  been  established 
satisfactorily  by  existing  literature  to  be  at  least  6 bits  per  picture  element  with  8 bits  pre- 
ferred. No  similar  guideline  applicable  to  resolution  has  been  established.  In  order  to  derive 
these  data  on  resolution,  tests  were  planned,  conducted,  and  analyzed  as  described  in  this 
report. 

The  resolution  level  needed  in  the  RMDS  visual  channel  will  be  that  which  will  enable 
the  radiologist  at  the  receiving  RMDS  station  to  perform  satisfactorily  his  evaluations  of  the 
RMDS-transmitted  material.  This  means  the  radiologist  must  be  able  to  diagnose  advanced 
pathological  conditions  and  injuries  with  the  RMDS-transmitted  material  in  a manner  sub- 
stantially the  same  as  he  would  if  the  original  X-ray  films  were  used.  It  should  be  noted  that 
the  resolution  requirement  may  not  be  as  great  as  would  be  the  case  of  making  more  definitive 
interpretations  typical  of  normal  clinical  screening  practices. 


TEST  OBJECTIVE 

It  is  important  for  technical  and  cost  reasons  not  to  overdesign  RMDS  with  respect 
to  gray  scale  and  resolution.  Since  data  already  available  establish  satisfactorily  gray-scale 
requirements,  the  tests  reported  herein  are  limited  to  consideration  of  resolution  require- 
ments. Specifically,  the  objective  will  be  to  determine  just  how  much  resolution  is  needed 
for  the  radiologist  to  make  correct  interpretations  of  X-ray  pictures  showing  injuries  or 
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advanced  pathological  conditions.  Al  ter  this  resolution  level  is  det'ined,  it  will  then  be 
possible  to  design  the  various  components  ol  the  system  to  ensure  that  the  level  is  main- 
tained operationally  on  the  displays  at  the  receiving  station. 
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TEST  APPROACH  j 

I 

In  order  to  determine  the  level  of  resolution  suitable  for  RMDS  usage,  radiologists 
were  asked  to  evaluate  radiographic  images  presented  to  them  on  a video  monitor  at  several 
levels  of  resolution.  For  the  tests,  10  radiographs  were  selected  and  divided  into  2 test  groups. 

Each  radiograph  was  imaged  on  a special,  digital,  closed-circuit  television  system  with  a 
defocused  camera  lei.s.  The  degraded  electronic  images  were  recorded  on  video  tape  in  a 
sequence  of  images  from  poorest  to  best.  The  tape  was  then  used  to  present  the  images  on  a 
television  screen  in  the  desired  test  sequence.  The  pre.sented  images  were  similar  to  those 
which  would  be  obtained  from  an  RMDS  SSTV  system  with  respect  to  content,  size,  orien- 
tation, and  contrast. 

The  radiologists  were  tested  individually  using  the  video  sequence  and  then  the 
original  X-ray  film.  Members  of  the  test  group  I were  asked  to  evaluate  diagnoses  already 
made  with  respect  to  the  test  items;  members  of  the  test  group  2 were  asked  to  make  their 
own  interpretations  of  the  test  items.  Neither  case  histories  nor  clinical  data  were  available 
to  either  group.  The  diagnoses  and  evaluations  were  analyzed  subsequently  and  the  re- 
sponses to  the  degraded  images  were  compared  with  the  responses  to  the  Him  readings. 

Conclusions  on  resolution  requirements  were  obtained  from  this  anaJy.sis. 

The  remainder  of  this  report  is  divided  into  5 sections.  The  next  section  provides 
a detailed  discussion  of  the  equipment  and  other  experimental  apparatus  and  materials. 

Then,  in  order,  following  sections  describe  the  procedures  used  for  the  experiment,  document 

the  results  of  the  experiment  including  the  analysis  of  data,  discuss  the  experimental  results,  , 

and,  finally,  present  conclusions  and  recommendations  derived  from  the  experiment,  the 

latter  including  specific  resolution  requirements. 

TEST  MATERIALS  AND  EQUIPMENT 

GENERAL 

Various  alternatives  were  considered  for  presenting  the  degraded  radiographic  images 
for  evaluation.  These  included  a rear-screen  video  projection  system  with  defocused  optics, 
a closed-circuit  television  system  (('(TV)  with  various  band-limiting  filters,  and  a ('CTV  with 

disc-  or  tape-recorded  images.  The  rear-screen  projection  system  was  eliminated  because  the  « 

image  pre.sented  with  such  a .system  was  found  to  he  totally  noise-free  and  not  representative 
of  a realistic  electronic-image  output.  The  (’(’TV  system  with  tape-recorded  images  was 
selected  because  suitable  equipment  to  implement  the  other  approaches  was  not  available. 

RADIOGRAPH  SELECTION 

It  would  have  been  ideal  to  have  large  numbers  of  test  images  and  radiologists  for  use 
in  the  tests  in  the  interest  of  representative  sampling.  However,  limited  personnel  resources 
made  this  approach  impractical.  Further,  the  difficulties  anticipated  in  obtaining  time  from 
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radiologists  plus  the  adverse  effeets  of  over-long  tests,  set  a limit  of  one  hour  on  the  test 
period  of  each  radiologist.  This  meant  that  only  a very  limited  number  of  X-ray  photographs 
could  be  used  and  that  these  would  have  to  be  selected  with  special  care  so  as  to  be  a 
representative  sample  and  provide  an  adequate  testing  range.  Because  pulmonary  diseases 
are  the  most  prevalent  of  shipboard  maladies,  half  of  the  films  selected  for  evaluation  were 
chest  films.  This  decision  was  supported  further  by  the  fact  that  the  chest  film,  because 
of  its  physical  size  and  considerable  range  of  detail,  was  expected  to  be  the  most  demanding 
type  of  radiograph  (in  terms  of  information-transfer  requirements)  to  be  transmitted. 

A detailed  evaluation  of  a number  of  possible  candidate  films  was  performed  by  a 
senior  radiologist  on  the  study  team.  This  evaluation  resulted  in  a selection  of  10  radiographs 
for  use  in  the  test.  Appendix  A contains  photographs  of  these  10  test  films  and  table  1 lists 
them  along  with  an  a priori  evaluation  of  the  probable  diagnoses.  The  series  included  5 chest 
films  of  which  4 were  abnormal  and  one  was  nonnal,  2 lateral  skull  films  of  which  one  was 
normal  and  one  had  a fracture,  one  sinus  film,  one  pelvic  film,  and  one  showing  a fracture  of 
a major  bone.  Additional  details  on  the  cases  and  (he  diagnoses  will  be  provided  later  on  in 
this  report. 


TABLE  1.  SUMMARY  OF  TEST  FILMS. 


Case  Subject  (View)  Principal  Diagnoses 


Bone  (femur) 

Major  fracture  with  displacement 

Chest  (AP)* 

Atelectasis 

Possible  pleural  effusion 

3 

CTiest  (AP) 

Multiple  injuries 

Multiple  rib  fractures 

Fracture  of  scapula 

Hematoma 

Possible  hemothorax 

4 

Chest  (AP) 

Normal 

S 

Chest  (AP) 

Active  tuberculosis 

6 

Pelvis  (PA)** 

Fracture  (subtle)  of  right  pubic  ramus 

7 

Sinus 

Sinusitis 

8 

Skull  (lateral) 

Normal 

9 

Chest  (AP) 

Pneumothorax 

10 

Skull  (lateral) 

Multiple  fracture 

• Anterior  to  posterior 
••  Posterior  to  anterior 
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( ASl-  ONF,  BONF  \VH  H I RAC  TURF 


I'his  case  was  tliat  ol  a I'cnuir  Iracturc  at  approximately  mid-length.  The  fracture 
sliowed  a displacement  ol  approximately  one  inch  and  included  one  visible  bone  chip. 

CASi:  TWO,  c HKS 1 \vn  h a i i i.i;c‘  i a.sis 

The  radiograph  show'ed  a pronounced  light  area  in  the  lower  20  percent  of  the  lower 
left  lung.  This  area  was  clearly  abnormal  and  could  have  represented  either  atelectasis  or 
hemothorax,  of  which  the  former  was  deemed  correct.  No  indications  resembling  pneumonia 
were  present  in  the  film. 

( ASH  THRHH,  ClIliST  VVITII  MASSIVH  INJURIHS 

This  radiograph  was  of  a subject  who  had  suffered  significant  injuries  to  the  right 
shoukler  and  lung  areas  in  an  auto  accident.  Specifically,  the  right  scapula  had  been  fractured 
and  ilisplaced.  but  the  fracture  itself  was  not  obvious  and  only  the  position  of  the  scapula 
gave  the  clue.  The  light  region  in  half  of  the  right  lung  area  indicated  a hemotoma  or  a pos- 
sible hemothorax  condition.  The  film  also  clearly  indicated  an  anomaly  of  the  ribs  due  to 
their  peculiar  shape.  A very  close  examination  of  the  ribs  indicated  that  at  least  2 were 
fractured.  Diagnosis,  however,  was  made  difficult  because  the  film  in  that  area  was  ver>'  poor 
due  to  local  overexposure.  Atelectasis  was  not  a reasonable  diagnosis;  however,  as  the  film 
was  taken  with  the  patient  in  a prone  position,  it  was  possible  to  make  a misdiagnosis  of  that 
coiulition. 

CASH  HOUR.  CUHST  (NORMAL)  J 

I his  film  was  of  a normal  female  chest  with  no  visual  cues  which  could  have  been 
mistaken  for  any  of  the  given  diagnoses. 

( ASH  riVH..  ClIHST  WITH  ACTIVH,  TUBERCULOSIS 

In  this  film,  the  upper  lobe  of  the  right  lung  contained  radiologic  opacifications,  V 

generally  beneath  rib  structures.  The  shape  and  structure  of  these  opacifications  indicated 
active  tuberculosis  but  were  also  suggestive  of  bacterial  pneumonia  or  pleural  plaque.  Neither 
of  these  latter  2 diagnoses  was  correct. 


CASE  SIX.  PELVIS  WITH  FRACTURE  OF  RIGHT  PUBIC  RAMUS 

The  film  showed  a very  subtle  fracture  of  the  right  pubic  ramus.  Although  there 
were  no  clear  fracture  lines,  the  indications  were  slight  gradations  of  gray  scale  and  detail,  and 
the  shape  of  the  ramus.  There  was  possibility  of  misdiagnosis  of  an  intertrochaneric  fracture 
of  the  left  femoral  neck  due  to  the  fact  that  a line,  representing  the  edge  of  the  fatty  tissue 
of  the  buttocks,  crossed  the  femoral  neck  and  gave  the  appearance  of  a fracture.  This  misdiag- 
nosis would  be  rejected,  however,  upon  careful  examination,  since  this  line  could  be  seen  to 
extend  outside  the  femoral  neck. 

CASE  SEVEN.  SINUS  WITH  SINUSITIS 

The  film  for  this  case  showed  a significant  opacification  of  the  left  portion  of  the  face. 
This  could  be  attributed  to  either  sinusitis  or  swelling  of  soft  tissue.  Due  to  the  lack  of  a 
sharp  demarcation  of  the  opaque  area,  sinusitis  was  concluded  to  be  the  most  probable  diag- 
nosis, barring  any  other  clinical  data  to  support  the  diagnosis.  No  true  indication  of  facial 
fracture  appeared;  however,  in  the  case  of  swelling  of  soft  tissue  due  to  a contusion,  this 
could  be  a logical  possibility. 


CASE  EIGHT.  SKULL  (NORMAL) 

This  film  of  a lateral  view  of  the  skull  showed  a vascular  structure  running  downward 
from  the  top  central  portion  of  the  skull  which  might  appear  to  be  a major  fracture  under 
limited  resolution  conditions  In  fact,  there  was  no  fracture  of  any  kind.  The  pineal  body  on 
this  film  was  calcified,  and  thus  visible,  but  not  displaced. 


CASE  NINE,  CHEST  WITH  PNEUMOTHORAX 

This  film  showed  a sharp-edged,  but  subtle  pneumothroax  of  the  left  lung.  No  symp- 
toms were  present  to  indicate  any  of  the  otiter  listed  diagnoses. 


CASE  TEN,  SKULL  WITH  FRACTURE 

This  lateral  film  showed  numerous  long  (up  to  5 cm)  and  distinct  fractures  located  in 
the  center  and  back  of  the  skull.  The  pineal  body  was  not  calcified,  and  as  such,  the  pineal 
displacement  diagnosis  was  meaningless. 
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EQUIPMENT 


rile  equipment  employed  in  this  experiment  consisted  oT  the  hardware  necessary  to 
tape  record  and  subsequently  display  the  selected  radiographic  images.  The  main  elements 
were:  a television  came'a.  a logarithmic  atiiplitier.  a video  digitization  and  analog  conversion 
system,  a video  tape  recorder,  and  display  monitors.  Figure  1 illustrates  the  hardware  system 
configuration  used  to  develop  the  test  tape  for  the  experiment. 


TELEVISION  CAMER.A  SYSTEM 

Two  different  camera  units  were  employed  to  make  the  test  ta.  e.  For  8 of  the  10 
radiographs,  a C'ohu  “1  20  camera  with  a .Model  7900  remote  controller  unit  was  employed. 
This  is  a 32-MHz.  .xZ.^-line.  vidicon  television  system.  The  bandwidth  was  reduced  to  8-.MHz 
(-3-dB  point)  by  a plug-in  filter  in  the  camera.  The  measured  resolution  for  this  system  was 
b50  horizontal  lines  per  picture  height,  which  agreed  with  the  calculated  resolution  for  these 
conditions.  The  Vidicon  used  in  the  camera  was  an  RCA.  model  8.341  A,  1-inch,  separate- 
mesh  type. 

Because  the  C'ohu  71  20  was  not  available  when  the  last  2 radiograph  serii  were  made, 
a minicon  camera  with  essentially  the  same  characteristics  was  substituted.  A Nikkor  28-mni, 
f'1.8.  35-mm  format  lens  was  used  with  both  cameras. 


I.OCARITHMIC  AMPLIFIER 


The  range  of  optical  densities  inherent  in  radiographs  is  over  300  to  1 . while  the  range 
of  densities  of  interest  is  estimated  to  be  1 50  to  I ; however,  even  the  latter  dynamic  range  is 
beyond  the  capabilities  of  the  television  system  and  video  recorder.  In  particular,  if  a linear 


f igure  1 . Hardware  system  for  producing  test  tapes. 
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response  to  the  film  density  is  employed,  much  of  the  image  gradation  information  of  inter- 
est in  the  darker  portions  of  the  radiograph  is  lost  in  the  system  noise.  Since  the  information 
in  the  light  portions  of  the  image  is  usually  of  less  importance,  the  response  in  this  region 
can  be  sacrificed.  For  this  reason,  a specially-designed  logarithmic  amplifier  was  fabricated 
and  used  to  compress  the  dynamic  range  (response)  in  the  liglit  portions  of  the  radiographs 
and  expand  the  dynamic  range  in  the  dark  portions  of  the  radiographs.  This  amplifier  has  a 
bandwidth  of  10  MHz  and  provides  response  varying  from  linear  to  15  dB  of  adjustable 
compression  in  the  light  portion. 


VIDEO  DIGITIZATION  AND  ANALOG  CONVERSION  SUBSYSTEM 

It  was  desired  to  provide  images  on  the  test  tape  to  be  representative  of  digitally 
processed  (quantized)  images.  To  do  this,  an  existing  real-time  video  signal  digitization, 
frame  storage,  and  digital-to-analog  conversion  subsystem  was  used  to  produce  the  test  tapes. 
This  system,  identified  in  figure  1,  digitizes  and  stores  in  real  time  a frame  of  data  at  6 bits 
per  picture  element,  reads  frame  data  from  memory,  and  converts  these  data  to  quantized 
(unfiltered)  analog  form  as  a 525-line  NTSC  standard  2-to-l  interlaced-scan  signal.  The  out- 
put image  contains  975  picture  elements  in  the  horizontal  direction  and  forms  an  image 
with  the  commercially  standard  4-to-3  aspect  ratio. 

It  should  be  noted  that  this  approach  to  simulating  the  RMDS  visual  channel  neces- 
sarily lacks  full  fidelity,  particularly  with  respect  to  noise.  The  differences,  as  will  be  ex- 
plained, made  the  test  situation  less  demanding  than  the  RMDS  with  respect  to  reading  through 
noise.  The  RMDS  will  transfer  a single  image  frame  to  the  receiving  terminal  where  it  is  stored 
together  with  all  system  noise  accumulated  in  transmission  and  processing.  The  stored  image, 
including  its  static  noise  buildup,  is  read  out  of  storage  30  times  per  second  for  display.  In  the 
test  system,  on  the  other  hand,  a new  frame  was  recorded  on  tape  each  1/30  of  a second  so 
system  noise  was  randomized  from  frame  to  frame  on  playback.  The  operational  difference 
is  that,  in  viewing  the  actual  RMDS  display,  random  noise  from  the  transmission  and  process- 
ing becomes  an  integral  part  of  the  stored  image  whereas,  with  the  test  display,  the  noise  forms 
a random  pattern  from  frame  to  frame.  At  the  30-cycle  frame  ratio,  the  noi.se  will  not  summate 
to  a perceptible  level  for  the  observer  so  the  system  noise  which  appears  randomly  on  the 
frame  will  not  be  confused  with  picture  noise.  On  the  other  hand,  in  interpreting  the  RMDS 
image,  the  user  will  see  system  noise  displayed  at  fixed  positions;  this  will  tend  to  interfere  with 
picture  interpretation.  (The  effect  would  be  similar  to  over-printing  a black  and  white  picture 
with  ink  spots.)  The  magnitude  and  significance  of  this  difference  with  respect  to  noise 
between  the  test  and  actual  RMDS  displays  cannot  be  determined  nor  assessed  at  this  time, 
but  conservative  interpretation  of  the  experimental  findings  is  advisable  for  the  above  reasons. 


VIDEO  TAPE  RECORDER 

An  IVC  20(X’,  cassette,  video-tape  recorder  was  used  for  recording  and  playback  of  the 
test  tape.  The  resolution  of  this  system  for  record  and  subsequent  playback  was  measured  at 
over  550  lines. 


% 
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DISPLAY  MONITOR 


The  television  monitor  used  to  display  the  test  images  was  a GBC  CTTV  Corporation, 
Model  MV-17,  with  a lO-Mllz  bandwidth  and  a I 7-ineh  screen.  Front-panel  brightness  and 
contrast  controls  were  accessible  to  the  radiologists  lor  use  during  the  tests. 


RECORDINGS 

Because  it  was  desired  to  limit  the  participation  of  each  radiologist  to  one  hour,  only 
10  radiographs  at  each  of  5 levels  of  resolution  were  used.  This  represented  a total  of  50 
images  (40  TV  and  10  film)  to  be  evaluated  by  each  radiologist.  In  order  to  ensure  agreement 
between  test-image  numbers  and  the  radiologist  response  sheets,  each  radiographic  image  on 
the  test  tape  was  preceded  by  an  image  of  its  respective  number.  The  radiograph  images 
themselves  were  recorded  in  such  a manner  that  comparable  fields  of  view  and  orientations 
were  present  for  all.  For  the  4 video-image  recordings  of  each  radiograph,  the  resolution  was 
improved  for  each  succeeding  image  with  the  last  image  at  or  near  the  best  resolution  of 
which  the  television-recorder  system  was  capable.  The  best  television  recorder  system  resolu- 
tion which  was  measured  was  about  550  lines,  using  ElA  test  charts  at  the  appropriate  view- 
ing distance. 

In  order  to  determine  accurately  the  degree  of  resolution  being  recorded  on  the  tape, 
this  resolution  was  measured  for  each  of  the  40  images.  To  do  this,  a bar  pattern  of  known 
spatial  resolution  was  viewed  in  place  of  the  film  and  all  of  the  resolution-innuencing  factors 
of  the  camera  configuration  were  held  constant.  The  camera  optics  were  then  defocused  (if 
appropriate)  until  the  bars  representing  the  desired  spatial  frequency  appeared  at  about  half 
their  best  contrast  ratio.*  At  this  setting,  the  bars  for  the  next  higher  resolution  were  com- 
pletely defocused  while  the  bars  for  the  next  lower  resolution  were  generally  sharp.  With  the 
bar  resolution  set,  the  bar  pattern  was  replaced  by  the  radiograph  and  the  desired  video-tape 
recording  was  made.  Just  prior  to  making  each  recording,  the  digital  system  was  tested  to 
ensure  that  the  video  signal  was  optimum;  that  is,  the  full  dynamic  range  of  the  digital  system 
was  being  used. 

Table  2 is  a listing  of  the  recordings  which  were  made  and  the  corresponding  resolu- 
tion data.  The  calculated  resolution,  in  television  lines  per  picture  height,  is  based  upon  the 
measured  resolution  at  the  film,  the  measured  horizontal  field  of  view,  and  the  4-to-.J  aspect 
ratio  of  the  television  system  which  was  used.  Appendix  B of  this  report  contains  photo- 
graphs of  the  recorded  images  used  in  the  tests. 


* This  adjastment  was  performed  using  the  visual  display  after  passing  the  video  througli  the  recorded 
electronics 
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TABLE  2.  RECOKIJING  PARAMETERS. 


!] 


Resolution 

Case 

Subject 

Recording 

(Lp/mm) 

TV  Lines 

Other 

1 

Bone  fracture 

1 

0,157 

66 

F/2.8 

2 

0.315 

132 

lloriz  field  of  view,  1 1 in. 

3 

0.63 

264 

60-sec  recording 

4 

1.0 

419 

2 

Chest  with  collapsed 

5 

0.079 

43 

F/2.8 

left  lower  lobe  (AP) 

6 

0.157 

85 

Recordings  5, 6,  at  90  sec; 

7, 8 at  80  sec 

7 

0.315 

171 

Horiz  field  of  view  14.25  in. 

8 

0.63 

342 

3 

Chest  with  multiple 

9 

0.079 

50 

F/5.6 

injuries  (AP) 

10 

0.157 

99 

90-sec  recordings 

11 

0.315 

198 

Horiz  field  of  view  i6.5  in. 

12 

0.63 

396 

4 

Normal  female 

13 

0.079 

47 

F/2.8 

chest  (AP) 

14 

0.157 

94 

90-sec  recordings 

15 

0.315 

189 

Horiz  field  of  view  15.75  in. 

16 

0.63 

378 

5 

Chest  with  active 

17 

0.079 

47 

F/4 

tuberculosis  (AP) 

18 

0.157 

94 

90-sec  recordings 

19 

0.315 

189 

Horiz  field  of  view  15.75  in. 

20 

0.63 

378 

6 

Pelvis  with  fracture 

21 

0.157 

80 

F/2.8 

of  right  pubic  ramus 

22 

0.315 

160 

90-sec  recordings 

(PA) 

23 

0.63 

321 

Horiz  field  of  view  13.375  in. 

24 

1.0 

521 

Reset  log  amplifier  to  com- 
press dynamic  range  and 
bring  out  dark  areas  while 
sacrificing  bright  areas. 

7 

Sinus  with  sinusitis 

25 

0.315 

100 

F/ll 

26 

0.42 

134 

90-sec  recording 

27 

0.63 

201 

Horiz  field  of  view  8.375  in. 

28 

1.2 

303 

Reset  log  amplifier  for  broad- 
er range  of  gray  scales  in 
midrange  area. 

8 

Normal  lateral  skull 

29 

0.157 

70 

F/4 

30 

0.315 

141 

90-sec  recordings 

31 

0.63 

282 

Horiz  field  of  view  1 1.75  in. 

32 

1.2 

537 

9 

Chest  (AP) 

33 

0.079 

41 

MINICON  Camera 

pneumothorax  (at 

34 

1.157 

82 

F/2.8 

expiration) 

35 

0.315 

164 

90-sec  recordings 

36 

0.63 

327 

Horiz  field  of  view  1 1.625  in. 

Reset  log  amplifier  for 
uniformity  over  dynamic 

range. 

10 

Skull  (lateral)  with 

37 

0.157 

70 

MINICON  Camera 

multiple  fractures 

38 

0,315 

141 

F/2.8 

39 

0.63 

282 

90-sec  recordings 

40 

1.2 

537 

Horiz  field  of  view  1 1 .75  in. 

PROCEDURES 


TEST  CONDITIONS 

The  tests  pert'oniiecl  by  the  radiologists  were  eondiieted  at  the  Naval  Regional 
Medical  Center,  San  Diego,  in  an  unoeeupied  olfice.  The  television  monitor  was  placed  on 
a desk  with  the  tape  recorder  on  a nearby  table,  the  radiologist  was  seated  at  the  desk,  approx- 
imately 2 feet  (bO  cm)  from  the  monitor  screen,  but  free  to  move  closer  to  or  further  from 
the  monitor  for  comfortable  viewing.  The  desk  space  between  the  monitor  and  radiologist 
was  available  as  a working  surface  for  marking  the  test  data  sheets.  The  background  light  in 
the  room  was  controlled  with  Venetian  blinds  to  provide  good  contrast  for  viewing  the 
monitor. 


TEST  SUBJECTS 

Eight  practicing  radiologists  participated  in  the  tests.  They  varied  in  background 
from  a resident  with  one  year  of  experience  to  2 staff  members,  each  with  6 years  of  exper- 
ience. Ihe  average  experience,  including  residency,  was  3..‘i  years.  The  8 radiologists  were 
divided  into  2 comparable  groups  of  4 each  for  2 different  test  approaches.  The  experience, 
position,  and  tests  for  each  radiologist  are  listed  in  table  3. 


l ABLE  3 DATA  ON  PARTICIPATING  RADIOLOGISTS. 


Radiologist 

Position 
at  NRMC 

Years  of 
Experience* 

Test 

(Group  I) 
Diagnoses 
Evaluation 

(Group  2) 
linage 

Diagnosis 

A 

Resident 

2.5 

X 

B 

Resident 

3.0 

X 

C 

Staff 

4.0 

X 

D 

Staff 

6.0 

X 

I- 

Resident 

1.0 

X 

F 

Resident 

1.3 

X 

(f 

Staff 

4.0 

X 

II 

Staff 

6,0 

X 

* Includes  residenev 


TESTING  SESSIONS 

The  testing  sessins  were  conducted  during  normal  working  hours.  A test  supervisor 
was  present  for  the  entire  test  sequence  to  ensure  that  the  test  equipment  was  ready,  to 
operate  the  video  recorder,  to  provide  the  radiologist  with  the  films  for  viewing,  and  to 
answer  any  procedural  questions. 

When  a radiologist  arrived  for  the  test,  he  was  informed  that  he  was  participating 
in  a data-gathering  experiment  and  was  asked  to  review  a set  of  instructions  concerning  the 
experiment  background  and  procedures.  After  clearing  up  any  procedural  questions,  the 
test  supervisor  then  started  the  test  with  the  video  tapes  and  continued  until  the  tape  was 
completed  and  the  test  data  sheets  were  filled  out.  (The  option  of  image  rerun  was  always 
open  to  the  radiologists  but  was  never  exercised.) 

Upon  completion  of  the  video  series,  the  radiologist  was  then  given  the  original  radio- 
graphs to  evaluate  with  a conventional  view  box.  These  were  presented  one  at  a time  with 
sufficient  time  allowed  for  evaluation  impressions  and/or  diagnoses.  The  test  was  completed 
when  a review  of  alt  the  radiographs  was  finished. 


APPROACHES 

Two  basic  test  approaches  were  employed.  In  one,  the  radiologists  (group  1)  were 
given  test  sheets  containing  previously  prepared  diagnoses  and  were  asked  to  evaluate  them. 
In  the  second  approach,  the  radiologists  (group  2)  were  asked  to  supply  their  own  diagnoses. 
The  two  test  approaches  are  referred  to  in  this  report  as  the  “Diagnoses  Evaluation  Test” 
(group  1 test),  and  the  “Image  Diagnosis  Test”  (group  2 test),  respectively. 


DIAGNOSES  EVALUATION  TESIS 

Of  the  8 radiologists  tested,  every  other  one.  starting  with  the  first,  participated  in 
the  Diagnoses  Evaluation  Tests.  Appendix  C contains  a copy  of  the  instructions  for  those 
participating  in  these  tests. 

Figure  2 is  a sample  of  the  test  sheet  used  in  the  Diagnoses  Evaluation  Tests.  One 
test  sheet  was  provided  for  each  of  the  50  test  images  which  were  evaluated.  The  diagnoses 
entered  on  the  test  sheets  were  the  same  for  each  test  image  prepared  from  a given  radiograph. 
The  instructions  directed  each  radiologist  to  evaluate  each  diagnosis  on  the  test  sheet  for  the 
test  image  to  which  it  pertained  and  to  base  his  evaluation  strictly  upon  what  he  saw  in  the 
test  image. 

The  sets  of  diagnoses  presented  in  the  test  series  were  originally  prepared  by  the 
radiologist  on  the  study  team.  The  diagnoses  included  in  each  set  were  those  which  would 
appear  to  be  reasonable  for  the  respective  radiographs  and  which  wouid  require  careful 
interpretation  of  the  radiograph  features  in  order  to  make  a correct  selection.  Table  4 lists 
the  cases  used  for  the  test  and  the  preassigned  diagnoses  in  the  order  presented  for 
evaluation. 


I 
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TEST  IMAGE  #21  □ 


IMAGE  SUBJECT;  PELVIS 

COMMENTS. 


Diagnosis: 


FRACTURE  OF  LEFT 
PUBIC  RAMUS 


Definitely 

Probably 

Cannot 

Probably 

Definitely 

Wrong 

Wrong 

Decide 

Correct 

Correct 

Diagnosis 


FRACTURE  OF  RIGHT 
PUBIC  RAMUS 


Definitely 

Probably 

Cannot 

Probably 

Definitely 

Wrong 

Wrong 

Decide 

Correct 

Correct 

Diagnosis: 

INTER  TROCHANERIC 
FRACTURE  OF  LEFT 
FEMORAL  NECK 


Definitely 

Probably 

Cannot 

Probably 

Definitely 

Wrong 

Wrong 

Decide 

Correct 

Correct 

Definitely 

Probably 

Cannot 

Probably 

Definitely 

Wrong 

Wrong 

Decide 

Correct 

Correct 

Definitely 

Probably 

Cannot 

Probably 

Definitely 

Wrong 

Wrong 

Decide 

Correct 

Correct 

Figure  2.  Sample  of  test  sheet  used  in  diagnoses  evaluation  tests. 
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TABLE  4.  DIAGNOSES  PRESENTED  EOK  I VALUA TION 
IN  THE  DIAGNOSES  EVALUATION  Ti:STS 


Case 

Subject 

Diagnoses  for  Evaluation 

I 

Bone 

Fracture* 

Normal 

2 

Chest  (AP)' 

Atelectasis* 

Hemothorax 

- Normal 

- Pneumonia 

- Pneumothorax 

3 

Chest  (AP) 

— Atelectasis 

— Fracture  of  scapula* 

- Hemothorax* 

Hematoma* 

- Rib  fracture(s)* 

4 

Chest  (AP) 

— Atelectasis 

Hemothorax 

— Normal* 

Pneumonia 

— Pneumothorax 

5 

Chest  (AP) 

Active  tuberculosis* 

Bacterial  pneumonia 

Normal 

— Pleural  plaque 

6 

Pelvis  (PA)“ 

Fracture  of  left  pubic  ramus 

— Fracture  of  riglit  pubic  ramus* 
Inter-trochaneric  fracture  of 
left  femoral  neck 

— Normal 

7 

Sinus 

— Facial  fracture 

— Normal 

— Sinusitis* 

Soft  tissue  swelling 

8 

Skull  (Lateral) 

Fracture 

- Normal* 

- Pineal  displacement 

9 

Chest  (AP) 

Atelectasis 

— Hemothorax 

— Normal 

— Pneumonia 

Pneumothorax* 

10 

Skull  (Lateral) 

- Fracture* 

Normal 

Pineal  displacement 

• Correct  or  potentially  correct  diagnosis 

* Anterior  to  posterior 
^ Posterior  to  anterior 
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IMACih  DIAGNOSIS  'H-SIS 


reiiuired  that  tlicy  provide  their  own  diagnoses  based  upon  what  they  saw  in  the  test  image. 
These  diagnoses  were  totally  unprompted  and,  t'urtlier,  recpiired  that  the  radiologist  indieate 
his  relative  eonfidenee  in  his  diagnosis. 

Appendix  1)  eontains  a eopy  ol  the  pretest  instruetions  provided  to  the  radiologists 
partieipating  in  this  second  test  series.  The  procedures  and  test  tape  lor  eondueting  this 
testing  were  the  same  as  those  lor  the  Diagnoses  Evaluation  I'ests. 

Figure  3 shows  a sample  of  the  test  sheet  used  in  the  Image  Diagnosis  Tests.  It  may 
be  noted  that  this  sheet  provides  tor  up  to  4 diagnoses  and  eonlldenee-level  indications 
ranging  from  low  to  high.  One  sheet  was  provided  for  each  of  the  50  images  evaluated. 

TEST  DATA  AND  ANALYSIS 


PRELIMINARY  CONSIDERATIONS 

In  planning  this  experiment,  it  was  assumed  that,  for  original  photographs  and  for 
individual  radiologists  of  roughly  equal  ability,  interpretations  for  each  ease  would  be  quite 
uniform.  This  did  not  turn  out  to  be  a valid  assumption.  Consider,  for  example.  Case  nine 
(see  table  51.  For  (he  diagnosis  of  ate/eefasis.  Radiologists  (’  and  D.  (he  2 most  experienced 
radiologists  in  the  group,  were  diametrically  opposed  in  their  evaluations.  Furthermore,  the 
4 radiologists  were  split  half-and-half  as  to  whether  or  not  aleleelasis  was  present,  while 
reailing  the  same  original  tilm.  This  lack  ot  agreement  occurred  in  other  instances  during 
the  testing. 

It  should  be  noted  that  the  original  films  were  interpreted  last  in  the  testing  period, 
and  that  the  test  ran  longer  than  the  allocated  hour.  This  means  that  the  film  readings  may 
have  been  rushed  in  some  eases  because  of  the  radiologist’s  need  to  return  to  more  pressing 
duties.  Also,  of  course,  the  readings  were  made  independently  and  without  benefit  of 
supplement  patient  history  and  symptomatic  information.  A review  of  results  obtained  in 
reailing  the  original  films  is  presented  later  on  in  this  report. 

DATA-ANALYSIS  APPROACH 

1 he  main  analysis  ellort  was  devoted  to  the  Diagnoses  I'valuation  bests.  The  Image 
Diagnosis  lesls  were  analyzed  to  supplement  the  main  analysis  effort  and  assist  in  drawing  ' 

eonelusions  for  each  radiographic  ease. 

In  analyzing  the  results,  the  performanee  of  each  radiologist  working  with  the  tele- 
vision images  was  compared  to  his  performanee  when  the  original  films  were  used.  In 
addition,  for  each  radiologist,  the  point  was  determined  where,  in  the  television  series,  the 
performanee  was  substantially  the  same  as  with  the  original  film.  This  method  established 
the  level  ol  resolution  needed  by  each  radiologist  in  order  to  perform  as  well  with  the 
television  images  as  with  the  original  films.  Hy  taking  into  aeeount  the  range  and  distribution 
of  responses,  as  well  as  situational  factors  for  each  ease,  the  analysts  derived  subjectively 
resolution  levels  which  appearetl  to  be  appropriate  for  the  radiologists  as  a group. 


1(. 


IMAGE  SUBJECT:  PELVIS 
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Figure  3.  Sample  of  test  sheet  used  in  image  diagnosis  tests. 
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QUALITATIVE  CATEGORIZATION  ANALYSIS 
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As  was  shown  in  figure  2.  tlie  test  sheets  used  in  the  Diagnoses  Evaluation  Tests  used  | 

5 categories  for  responses  to  each  preassigned  diagnosis.  These  were,  definitely  wrong,  , 

probably  wrong,  cannot  decide,  probably  correct,  and  definitely  correct.  In  the  qualitative 
rating  analysis,  the  evaluation  made  by  each  radiologist  (group  1 ) using  these  5 categories  on 
each  preassigned  diagnosis  with  the  television  series  was  compared  to  the  evaluation  made  by 

the  same  radiologist  working  with  the  original  Him.  fhe  performance  of  each  radiologist  on  j 

each  case  was  then  categorized  using  the  data  for  all  of  the  preassigned  diagnoses  for  the 
particular  case. 

COMPLETE  AGREEMENT  CATEGORY 

This  first  category  of  performance  represented  the  case  in  which  the  individual 
diagnosis  evaluations  agreed  completely,  on  a one-for-one  basis,  with  the  original  film-reading 

results.  For  example,  in  table  5.  the  evaluations  of  radiologist  A at  resolutions  of  0.3 1 5 ’ 

Ip/mm  and  0.63  Ip,  mm  would  be  placed  in  this  category  because  there  is  100  percent  agree- 
ment with  the  film-reading  results. 

MINOR  DISAGREEMENT  CATEGORY 

This  category  was  used  when  the  following  eonditions  were  met  in  comparing  the 
evaluations  of  the  diagnoses:  (a)  all  recorded  television-based  evaluations  agreed  “in 
direction"  with  the  film-reading  results;  and  (b)  no  “cannot  decide”  evaluation  of  any  diag- 
nosis was  given.*  By  this  definition,  the  results  of  radiologist  D for  0.3 1 5 Ip  mm  and  0.63 
Ip  mm  image  evaluations  of  table  5 would  be  categorized  as  “minor  disagreement.”  The 
results  of  radiologist  C for  the  same  resolution  would  also  be  categorized  as  “minor  disagree- 
ment” because  of  the  "cannot  decide”  evaluation  of  the  hemothorax  diagnosis. 

MODERATE  DISAGREEMENT  CATEGORY 

This  category  was  used  when  either  of  the  following  conditions  were  met  when 
comparing  the  diagnoses  evaluations:  (a)  the  evaluation  of  no  more  than  one  diagnosis  in  a 
direction  opposite  to  the  evaluation  made  from  film  reading  but  erring  conservatively;  and 
(b)  one  “cannot  decide”  evaluation  was  given. 

"Erring  conservatively”  as  used  here  is  defined  as  evaluating  an  “abnormal”  diagnosis 
as  probably  or  definitely  correct  when  the  subsequently  film-based  evaluation  resulted  in  a y 

“probably”  or  “definitely  wrong”  evaluation  for  that  diagnosis.  Clearly,  this  approach  has 
shortcomings.  In  a real-life  situation,  the  diagnosis  of  a condition  as  existing  when  in  fact 
it  did  not  exist  could  result  in  a prescription  of  treatment  which  might  he  detrimental  to  the 
patient’s  true  condition.  The  rationale  of  this  approach,  however,  is  that  a misdiagnosis  of  a 
condition  as  being  existent  would  likely  result  in  a more  extensive  investigation  of  the 
patient’s  condition.  Thus,  additional  examination  and  subsequently  derived  information 
should  correct  the  initial  erroneous  diagnosis. 

* If,  for  a d.agnosis,  the  film-reading  evaluation  was  “definitely  correct,”  a “probably  correct”  evaluation  of 
the  video  image  would  agree  in  direction.  Furthermore,  “probably  wrong”  or  “definitely  wrong”  evaluation 

would  be  con.sidered  to  be  in  the  opposite  direction  to  the  film-reading  result.  j 
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Hilscil  upon  lliCNO  critcri.i.  the  liiagno^cs  evaluations  hy  radiologist  C at  ().()3  Ip'mni 
in  table  5 were  in  the  "'inoderate  disagreement"  category  because  ot  the  "cannot  decide” 
evaluation  lor  the  hemothorax  diagnosis. 
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1 his  category  was  used  when  the  iliagnoses  evaluations  were  worse  than  the  "moderate 
disagreement"  category  ami.  s|iecirically.  when  any  ot  the  t'ollow'ing  conditions  were  found: 
la)  evaluation  ot  more  than  one  diagnosis  was  in  the  direction  opposite  to  that  of  the  film 
reading;  ( b)  the  recorded  television-based  diagnosis  was  such  as  to  evaluate  one  or  more 
abnormal  conditions  as  normal;  and  Ic)  when  more  than  one  "cannot  decide”  evaluation  was 


■MAJORITY  VOTE  ANALYSIS 

,A  routine  procedure  in  the  evaluation  of  radiographs  is  to  employ  a group  of  radiolo- 
gists in  the  film-reading  prt)cess.  I he  residting  diagnosis  represents  a consensus  of  opinion 
based  upon  the  combined  knowledge  and  the  interchange  of  viewpoints  among  the  radiolo- 
gists. In  Using  this  procedure,  the  combined  knowledge,  experience,  and  perceptions  of  the 
group  yield  a more  accurate  diagnosis  than  can  be  provided,  on  the  average,  hy  a single 
radiologist. 

In  the  tests  reporteil  upon  in  this  document,  however,  the  railiologists  worked  inde- 
pendently. It  was  thought  to  be  of  interest  to  determine  what  test  results  might  have  been 
achieved  had  the  radiologists  worked  as  a team.  In  order  to  obtain  some  measure  of  the 
team-approach  effectiveness,  a majority  vote  analysis  approach  was  used  to  analyze  selected 
data.  The  procedure  was  simply  to  determine  the  majority  opionion  of  the  4 group  I 
radiologist’s  evaluations  for  each  case,  diagnosis,  and  resolution  combination. 

The  rules  for  the  decision-making  process  of  the  majority  vote  analysis  were:  (a)  of 
the  4 given  evaluations  if  any  one  is  in  the  majority,  select  that  evaluation;  (b)  if  the  4 
evaluations  are  all  different,  assign  an  evaluation  of  "cannot  decide”;  and  (c)  if  the  4 separate 
evaluations  consist  of  2 evaluations  only,  then  the  majority  vote  evaluation  would  consist  of 
bt)th  listed  evaluations. 

I'his  majority  approach  is  very  simple  and.  as  such,  probably  has  shortcomings  as  a 
model  for  the  actual  staff-team  analysis  situation.  On  the  other  hand,  alternative  schemes 
such  as  weighting  the  votes  by  radiologist  experience  cannot  be  assured  as  being  more  accur- 
ate. This  is  for  the  reason  that  other  factors  such  as  personality,  specialties,  and  the  like 
woukl  also  innuence  the  group  result  and  would  he  too  complicated  to  be  modeled  in  this 
present  effort.  I hus.  the  selected  simple  majority  vote  approach  makes  the  fewest  assump- 
tions anil  is  likely  to  be  as  accurate  as  any  model  within  the  scope  of  this  study.  The  results 
of  the  majority  vote  analysis  for  the  Diagnoses  Evaluation  Test  are  included  in  Appendix  11. 


TEST  RESULTS 


I he  results  of  the  test  ;ire  presented  in  4 subsections;  radiographs  by  all  cases;  radio- 
graphs for  chest  cases  only;  television  images  by  cases;  and  television  images  by  diagnosis.  In 
the  following  paragraphs,  the  results  of  the  radiologists  examination  of  the  original  radio- 
graphs for  each  case  are  discussed.  No  discussion  is  presented  of  the  bone  fracture  case  since 


the  gross  nature  of  the  subject  material  did  not  provide  any  significant  data  other  than  the 
fact  that  all  radiologists  recognized  the  fracture. 


CASE  TWO,  CHEST  WITH  ATELECTASIS 

In  the  Diagnosis  Evaluation  Test,  all  radiologists  in  group  1 evaluated  the  "normal” 
and  "pneumothora.x”  diagnoses  as  "definitely  wrong”  and  the  "atelectasis”  diagnosis  as 
"definitely  correct.”  Three  of  the  4 radiologists  evaluated  "hemothorax”  as  "probably 
correct”  while  the  fourth  could  not  decide  on  that  diagnosis.  These  evaluations  rellect  the 
fact  that  both  atelectasis  and  hemothorax  present  basically  the  same  appearance,  a radio- 
graphically opaque  area.  Resolving  the  conflict  of  diagnosis  between  atelectasis  and  hemo- 
thorax would  have  to  make  use  of  more  careful  film  examinations  or,  more  probably,  patient 
history  or  symptomatic  data. 

Eor  the  Image  Diagnosis  Tests,  all  4 radiologists  in  group  2 indicated  a possible  pleural 
effusion;  fluid  in  the  pleural  cavity.  Two  of  the  radiologists  considered  atelectasis  a possibility. 

For  the  Diagnoses  Evaluation  Tests,  2 of  the  4 radiologists  in  group  1 believed  that 
"pneumonia”  was  probably  correct.  In  the  Film  Diagnosis  Tests,  only  one  of  the  4 group  2 
radiologists  indicated  a possibility  of  lower  left-lobe  (LLL)  pneumonia. 

From  an  analysis  of  the  majority  votes  of  the  group  I radiologists  in  the  Diagnoses 
Evaluation  Tests,  the  Tollowing  results  were  obtained:  Atelectasis,  "definitely  correct”; 
Hemothorax,  "probably  correct”;  Normal,  "definitely  wrong”;  and  Pneumothorax, 

"definitely  wrong.” 

In  the  case  of  pneumonia,  a split  decision  between  "probably  correct”  and  "probably 
wrong”  resulted  effectively  in  a "cannot  decide”  evaluation.  The  split  decision,  together  with 
the  fact  that  one  radiologist  in  group  2 made  a diagnosis  of  "LLL  pneumonia.”  indicates  that 
some  confusion  existed  as  to  the  pathological  nature  of  the  lower  left  lung  opacification. 

This  confusion  probably  could  not  have  been  resolved  from  the  film  image  and  thus  could 
not  have  been  resolved  from  the  television  images. 


CASE  THREE.  CHEST  WITH  MASSIVE  INJURIES 

Pre-experiment  analysis  by  the  project  radiologist  indicated  diagnoses  for  this  case  of 
multiple  rib  fractures,  fracture  of  the  right  scapula,  and  hematoma.  Hemothorax  was  con- 
sidered a possible  present  condition.  In  the  Diagnoses  Evaluation  Tests,  all  radiologists  in 
group  I agreed  with  these  evaluations  except  that  radiologist  C could  not  decide  the  existence 
of  scapula  fracture.  The  evaluation  responses  to  the  atelectasis  diagnosis  were  mixed  and 
varied  from  “probably  correct”  to  "probably  wrong”  and  included  a "cannot  decide.”  The 
fact  that  there  were  massive  chest  injuries  probably  made  this  latter  diagnosis  appear  reason- 
able. It  should  be  noted  that  the  participating  radiologists  were  instructed  to  base  their 
diagnostic  evaluations  only  upon  what  they  saw  in  the  pictures.  It  is  not  known,  however, 
whether  the  atelectasis  evaluations  were  based  upon  direct  visual  presentations  or  on  indirect 
consideration  of  the  fact  that  an  obvious  pulmonary  contusion  condition  was  present. 

A summary  of  the  diagnoses  by  radiologists  in  group  2.  the  Image  Diagnosis  for  this 
case,  is  pre.sented  in  table  6.  Note  that  the  rib  fracture  and  hemothorax  (ie.  pleural  effusion/ 
fluid)  conditions  were  correctly  diagnosed  by  all  radiologists,  fhe  scapula  fracture  was  seen 
by  only  2 of  the  4 participants,  and  2 radiologi.sts  diagnosed  a lesion  in  the  right  lung. 
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In  the  Diiiitnoscs  Fvaliiation  Tests,  all  radiologists  in  group  one  agreed  that  the  tihn 
was  probably  or  definitely  normal.  The  abnormal  diagnoses  presented  were  all  evaluated  as 
“definitely  wrong"  by  all  radiologists. 

In  the  Image  Diagnosis  Tests,  the  evaluations  of  the  radiologists  in  group  two  were: 
"normal"  (radiologist  F, );  “mass  left  Itilum”  (radiologist  F);  “centrally  enlarged  pulmonary 
arteries  (radiologist  G);  and  “left  hilar  density,  probably  large  pulmonary  artery  segment" 

( radiologist  H).  In  the  absence  of  obvious  symptomatic  conditions,  radiologists  F through  H 
presented  the  most  apparent  abnormality  they  could  find.  The  radiologists  generally  con- 
curred in  what  they  saw  as  the  most  apparent  abnormality  in  this  normal  radiograph. 


( ASF  FIVE.  ( HFST  WITH  ACTIVE  TUBERCULOSIS 

In  the  Diagnoses  Evaluation  Lests.  all  radiologists  in  group  one  agreed  the  film  was 
not  normal.  The  only  inconsistencies  in  the  evaluation  took  place  in  the  bacterial  pneumonia 
diagnosis.  For  this  diagnosis.  2 radiologists  could  not  decide  on  the  evaluation,  and  the 
remainder  evaluated  the  diagnosis  as  “probably  wrong”  and  “probably  correct.”  W'ith  regard 
to  the  pleural  placiue  diagnosis,  other  than  the  two  “cannot  decide”  evaluations,  the  other 
radiologists  indicated  that  this  iliagnosis  was  “probably  wrong.”  Three  of  the  4 radiologists 
believed  that  active  tuberculosis  was  “probably  correct”  and  one  could  not  decide. 

When  a majority  vote  analysis  was  conducted,  the  results  were:  active  tulx'rculosis. 
“probably  correct”;  bacterial  pneumonia,  “cannot  decide”;  normal,  “detlnitely  wrong”;  and 
pleural  plac|ue,  “cannot  decide.”  Clearly,  the  original  film  did  not  provide  sufficient  informa- 
tion to  definitely  diagnose  the  presence  of  bacterial  pneumonia  or  pleural  plaque. 

In  the  Image  Diagnosis  Fests,  all  of  the  radiologists  in  group  2 detected  the  2 right 
upper  lung  nodules;  of  the  radiologists  referred  either  to  tuberculosis  or  the  more  general 
diagnosis  of  granulomas  disease.  Other  diagnoses  referred  to  lower  middle  left-lung  densities. 

I he  fact  that,  in  this  test,  the  radiologists  did  not  specifically  mention  pneumonia  or  pleural 
plaque  mitigates  against  any  detrimental  conclusions  concerning  the  uncertainty  in  the 
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Diagnoses  Evaluation  Test  results  for  the  pneumonia  and  pleural  plaque  diagnoses  by  group  1 
'Ihe  presumably  correct  diagnosis  of  active  tuberculosis  was  evident.  The  uncertainty  in  the 
pneumonia  and  pleural  plaque  diagnoses  may  have  come  about  by  the  radiologists’  attempts 
to  find  very  early,  and  thus,  subtle  signs  of  these  diseases.  The  problem  could  have  been 
avoided  if  the  diagnoses  had  been  given  as  “advanced  bacterial  pneumonia”  and  “extensive 
pleural  plaque”  instead  as  presented. 


CASE  SIX,  PELVIS  WITH  FRACTURE 

The  Diagnoses  Evaluation  Test  results  indicate  that  all  radiologists  in  group  1 agreed 
that  the  right  pubic  ramus  was  fractured,  while  the  left  was  not.  One  difference  of  opinion 
occurred  concerning  the  inter-trochaneric  fracture  of  the  left  femoral  neck.  Radiologist  C 
failed  to  discern  the  fat  stripe  from  a fracture  line  and  thus  wrongly  evaluated  this  diagnosis 
as  “definitely  correct.” 

The  majority  vote  analysis  results  show  that  the  correct  diagnosis  of  a fracture  of  the 
right  pubic  ramus  would  be  evaluated  as  “definitely  correct.”  The  remaining  diagnoses  would 
be  evaluated  as  “definitely”  or  “probably  wrong.” 

In  the  Image  Diagnosis  Test  results,  3 of  the  4 radiologists  in  group  2 indicated  a diag- 
nosis of  fracture  of  the  right  ischium  and  pubic  ramus.  These  diagnoses  are  consistent  with 
the  pre-experimental  analysis  of  this  case  film.  One  radiologist  did  not  see  the  fracture,  ana 
instead,  listed  a diagnosis  of  a right  pelvic  mass  or  lytic  lesion. 


CASE  SEVEN,  SINUS  WITH  SINUSITIS 

In  the  Diagnoses  Evaluation  Test  results,  all  radiologists  in  group  I agreed  the  film 
subject  was  definitely  not  normal  and  that  the  sinusitis  diagnosis  was  either  “probably”  or 
“definitely  correct,”  and  that  facial  fracture  was  a wrong  diagnosis.  In  the  case  of  soft  tissue 
swelling,  the  radiologists  were  split,  with  one  half  evaluating  this  diagnosis  as  correct  and  the 
other  half  evaluating  the  diagnosis  as  wrong.  Thus,  the  majority  vote  analysis  yields  a 
“cannot  decide”  for  the  soft  tissue  swelling  diagnosis. 

In  the  Image  Diagnosis  Tests,  all  4 radiologists  in  group  2 indicated  some  form  of 
fluid  in  the  left  sinus,  with  2 radiologists  specifically  indicating  sinusitis.  Two  of  the  radiolo- 
gists also  indicated  a small  air  fluid  level  in  the  left  maxillary  sinus.  One  radiologist  also 
indicated  a diagnosis  of  a fracture  of  the  R/0  orbital  floor. 

The  soft-tissue  swelling  diagnosis  was  not  uniformly  evaluated  by  the  radiologists 
based  upon  their  film  readings.  Had  patient-condition  data  concerning  a possible  facial  con- 
tusion been  available,  the  evaluation  of  this  diagnosis  would  undoubtedly  have  been  uniformly 
the  same. 


CASE  EIGHT,  NORMAL  SKULL 

The  Diagnoses  Evaluation  Tests  for  this  case  produced  inconsistent  results  in  the  film 
reading.  The  fracture  diagnosis  was  evaluated  as  wrong  by  3 radiologists  in  group  1 and 
“definitely  correct”  by  the  fourth.  No  apparent  fracture  existed  on  the  film,  and  the  diagnosis 
of  the  fracture  may  have  been  a misdiagnosis  of  a supposed  hairline  fracture.  In  so  far  as  the 
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pineal  displacement  diagnosis  was  concerned,  one  radiologist  evaluated  this  as  “definitely 
wrong”  while  the  remainder  indicated  a “cannot  decide”  evaluation.  These  later  evaluations 
may  have  been  due  to  the  lack  of  a measurement  instrument,  as  the  pineal  was  clearly 
visible.  The  majority  vote  analysis  results  indicate  a fracture  as  “probably  wrong”  and  the 
normal  and  pineal  displacement  diagnoses  evaluations  as  “cannot  decide.” 

In  the  Image  Diagnosis  Tests,  2 of  the  4 radiologists  in  group  2 evaluated  the  film  as 
normal.  The  remaining  2 noted  carotid  calcification,  with  one  radiologist  further  noting 
dorsum  demineralization.  While  those  are  correct  diagnoses,  neither  is  considered  significant 
to  the  test. 


CASH  NINE.  CHEST  WITH  PNEUMOTHORAX 

In  the  Diagnoses  Evaluation  Tests,  the  radiologists  in  group  I agreed  perfectly  on 
their  evaluaiions  of  all  of  the  diagnoses,  except  atelectasis.  In  the  case  of  atelectasis.  2 
radiologists  evaluated  this  diagnosis  as  wrong,  while  the  remaining  two  evaluated  this  diag- 
nosis as  correct. 

Any  symptoms  of  atelectasis  must  have  been  extremely  subtle.  The  lack  of  com- 
ments by  the  radiologists  provided  no  indication  of  what  prompted  a positive  evaluation  for 
this  diagnosis.  One  radiologist  who  evaluated  atelectasis  as  “definitely  wrong”  provided  the 
following  comment;  “There  is  also  a pneumomediastinum  which  did  not  transmit  at  all.” 

Using  the  majority  vote  analysis  for  group  1 . all  diagnoses  presented,  except  the  correct 
one,  would  be  evaluated  a.s  “definitely  wrong.”  The  correct  diagnosis  would  be  evaluated  as 
“definitely  correct.”  In  the  Image  Diagnosis  Tests,  all  radiologists  in  group  2 diagnosed  the 
film  as  a case  of  pneumothorax. 

CASE  TEN,  SKULL  WITH  ERACTURE 

In  the  Diagnoses  Evaluation  Tests,  all  radiologists  in  group  I agreed  a fracture  defi- 
nitely existed.  In  that  the  pineal  was  not  calcified,  no  decision  was  made  on  the  pineal 
displacement  diagnosis.  The  Image  Diagnosis  Test  results  consisted  of  all  4 group  2 radiolo- 
gists diagnosing  multiple  fractures. 


COMPARISON  OF  FILM-READING  RESULTS  FOR  CHEST  CASES  ONLY 

NORMAL  DIAfiNOSES  FOR  CHEST  CASES 

Consider  the  results  of  the  Diagnoses  Evaluation  Tests  with  respect  to  the  preassigned 
normal  diagnoses  (Appendix  E).  In  this  case,  the  abnormal  films  (cases  2.  .‘i.  and  4)  are  all 
correctly  evaluated  as  such;  the  normal  diagnosis  is  evaluated  as  “definitely  wrong.”  In  case 
four,  in  which  there  was  no  abnormality,  all  radiologists  indicated  the  film  as  probably  or 
definitely  normal. 
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HEMOTHORAX  DIAGNOSES  FOR  ( HES T CASES 


In  the  4 chest  cases  where  hemothorax  was  a diagnosis  for  evaluation,  the  evaluation 
results  from  the  film  were  generally  consistent  among  the  radiologists.  The  cases  where 
hemothorax  was  not  the  correct  diagnosis  (4  and  9),  the  diagnoses  were  consistently  evaluated 
as  “definitely  wrong.”  In  case  three,  where  hemothorax  was  possibly  correct,  all  radiologists 
evaluated  the  diagnosis  as  “probably  correct.”  In  case  two,  where  the  actual  diagnosis  was 
atelectasis,  3 of  the  4 radiologists  felt  that  pneumothorax,  which  is  visually  similar  to 
atelectasis,  was  “probably  correct.” 


PNEUMOTHORAX  DIAGNOSES  FOR  CHEST  CASES 

Three  cases  included  the  diagnoses  of  pneumothorax  ( 2,  4,  and  9).  All  radiologists 
in  all  3 cases  were  able  to  accurately  evaluate  the  diagnosis. 


ATELECTASIS  DIAGNOSIS  FOR  CHEST  CASES 

Four  cases  included  the  diagnosis  of  atelectasis  (2,  3,  4.  and  9).  with  one  actual  case 
( 2)  of  atelectasis.  The  radiologists  were  in  agreement  and  accurate  in  their  evaluation  of  the 
atelectasis  diagnosis  for  case  2 and  in  the  normal  chest  film,  case  4.  In  the  chest  cases  with 
pneumothorax  (9)  and  the  case  with  massive  injuries  (3),  the  radiologists  disagreed  in  their 
diagnoses.  In  these  cases,  the  majority  vote  analysis  results  were;  case  three  Atelectasis, 
“definitely  correct”;  case  nine  Atelectasis,  “definitely  wrong.”  Thus,  with  the  majority  vote 
analysis,  the  only  discrepancy  was  in  Ca.se  three  where  mas.sive  injuries  existed  and  atelecta.sis 
was  not  considered  a correct  diagnosis. 


PNEUMONIA  DIAGNOSES  FOR  CHEST  CASES 

The  pneumonia  diagnosis  was  evaluated  in  4 cases  (2.  4,  5,  and  9)  and  was  not  the 
correct  diagnosis  in  any  of  these  cases.  In  the  cases  of  the  normal  chest  film  (4)  and  the 
pneumothorax  case  (9)  all  radiologists  indicated  that  pneumonia  was  “definitely  wrong.” 

In  cases  where  active  tuberculosis  (5)  and  atelectasis  (2)  were  the  actual  diagnoses,  the  rauiolo- 
gists  provided  confiicting  viewpoints  in  their  evaluations.  From  these  latter  results,  it  seems 
evident  that  without  further  supporting  information,  visual  symptoms  of  bacterial  pneumonia 
as  typified  by  ca.ses  two  and  five  are  not  amenable  to  accurate  radiographic  interpretation. 


TV-BASED  IMAGE  RESULTS  BY  CASES 

The  following  paragraphs  di  cuss  the  results  of  the  radiologists’  examination  of  the 
television  images  during  the  testing.  Utilizing  the  (|ualitative  rating  .ipproach  disc.isscd  earlier, 
the  data  from  the  Diagnoses  Evaluation  Tests  of  group  I were  analyzeil.  Appendix  H presents 
the  results  for  each  level  of  resolution,  including  the  majority  vote  ilata.  ( Because  of  the 
straightforward  results  obtained  for  the  bone  fracture  (case  one)  it  is  not  summarized  in 
Appendix  H.) 


CASK  ONI..  UONl  1 RAC TURi: 


liven  at  the  lowest  resolution  ot  0. 157  Ip/inm,  all  radiologists  were  able  to  evaluate  and 
diagnose  the  suhjeet  image  as  a t'raeture.  At  the  third  level  of  resolution.  ().C).7  Ip 'mm.  one 
group  3 radiologist  in  the  Image  Diagnosis  Test  indieated  that  the  I'raeture  involved  a small 
fragment  between  the  major  fragments.  This  bone  fragment  measured  about  6.5  mm  by  .3. .3  mni 
and  was  separated  from  the  main  fragment  by  about  O.X  mm  along  a eurved  line  the  length  of 
the  ehip.  This  length  lay  in  the  vertical  direction  of  the  television  image. 

System  resolution  good  enough  to  allow  recognition  of  bone  fragments  should  also  be 
good  enough  for  positive  recognition  of  fractures.  Thus,  fracture  lines  with  widths  on  the 
order  of  the  system  resolution  tin  line  pairs  per  millimeter)  should  be  readily  detectable  in  tllms 
transmitted  with  the  RMDS  system.  This  statement  must  be  qualified  by  the  assumptions  that 
the  fracture  is  shown  roughly  in  the  vertical  direction  on  the  television  monitor,  and  that  the 
length  of  the  fracture  is  greater  than  about  4 times  the  fracture  width. 


( ASF  TWO.  Clli:S  f WITH  ATELFCTASIS 

.At  the  lowest  resolution  of  0.079  Ip/mni.  all  of  the  radiologists  were  able  to  detemiine 
that  the  film  was  not  normal.  Furthermore,  based  upon  the  comments  from  the  Image  Diag- 
nosis Test  at  this  resolution,  all  radiologists  in  group  2 felt  that  effusion  in  the  lower  left  pleural 
was  possible.  (Confidences  ranged  from  low  to  medium.) 

As  the  resolution  was  improved,  all  radiologists  in  group  1 (the  Diagnoses  Evaluation 
Test)  coneJuded  that  the  atelectasis  diagnosis  was  “probably”  or  “definitely  correct.”  At  the 
second  level  of  resolution.  0. 1 57  Ip/mm.  in  the  Image  Diagnosis  Tests.  .3  of  the  4 radiologists  in 
group  2 had  indicatetl  atelectasis  as  a possihiliy.  with  a medium  to  medium-high  confidence 
level. 

In  both  the  Diagnoses  Evaluation  lest  and  the  Image  Diagnosis  Test,  the  pneumothorax 
diagnosis  was  either  discounted  or  not  mentioned.  Furthermore,  in  both  tests,  the  evaluations 
of  pneumonia  and  hemothorax  versus  atelectasis  were  not  unanimously  resolved  in  either  the 
films  or  the  television-based  images. 

The  majority  vote  results  for  the  Diagnoses  Evaluation  Tests  indicate  that  all  of  the 
resolution  levels  i)roduced  some  results  in  the  “serious  disagreement”  category.  On  the  other 
hand,  in  the  Image  Diagnosis  Test,  results  at  the  third  resolution  level,  ()..315  Ip/mm.  3 of  the 
4 group  2 radiologists  considered  atelectasis  a possibility  with  medium  to  high  confidenee.  The 
next  most  agreed-upon  diagnosis  was  pleural  scarring  (by  2 of  the  4 radiologists).  Also  it  must 
be  noted  that  all  8 radiologists  determined  this  case  was  not  normal  even  at  the  lowest  resolution 
level. 

Ba.seil  upon  the  test  results,  but  weighing  the  results  of  the  Image  Diagnosis  lest  more 
heavily,  the  0.315-lp/mm  resolution  would  appear  to  be  satisfactory  for  analysis  of  this  case, 
rinis,  at  this  resolution,  the  results  will  be  as  good  as  can  be  achieved  under  the  test 
circumstances. 


case:  itire;e,  ciie:.st  with  massive  in.iuries 

In  the  Diagnoses  Evaluation  Test,  and  at  the  third  resolution  level.  0.315  Ip/mm,  2 of 
the  4 radiologists  in  group  I indicated  a “cannot  decide”  on  two  diagnoses  and  reversed  their 
opinion  on  a third  diagnosis,  when  reading  the  original  railiograph.  .'\  thiril  railiologist 


indicated  a “cannot  decide”  evaluation  on  4 of  the  5 diagnoses.  The  fourth  radiologist  had 
nearly  perfect  agreement  with  his  radiograph  reading  results. 

In  the  Image  Diagnosis  Tests,  it  was  at  0.315-lp/mm  resolution  that  3 of  the  radiolo- 
gists in  group  2 defined  the  EKG  pad  on  the  left  chest  as  other  than  a nodule  in  the  left  lung. 
One  radiologist  continued  to  evaluate  this  EKG  pad  as  a nodule  even  at  the  best  television 
resolution  (0.63  Ip/mrn).  At  0.315-lp/mm  resolution,  the  radiologists’  comments  appear  to 
indicate  that  the  patient’s  recumbent  position  for  the  radiograph  was  not  obvious.  The 
obviously  misshaped  right  lung  was  apparent  but  its  source  was  not  defined. 

At  0.63-lp/mm  resolution,  the  qualitative  evaluations  of  the  group  I radiologists 
evaluations  were  divided  between  “moderate  disagreement”  and  “minor  disagreement,”  as 
given  in  table  2.  For  0.63-lp/mm  resolution,  and  for  the  Image  Diagnosis  Test,  half  (2)  of  the 
radiologists  in  group  2 indicated  hematoma  (in  equivalent  terms),  while  all  indicated  pleural 
effusion  of  the  right  lung.  Further,  2 of  the  group  2 radiologists  indicated  fracture  of  the 
right  ribs.  In  these  tests,  and  at  this  resolution,  none  of  the  group  2 radiologists  noted  the 
fracture  of  the  scapula,  although  when  reading  the  original  film,  2 did  note  this  diagnosis. 
However,  at  this  resolution,  all  4 group  I radiologists  indicated  “definitely  correct”  for  the 
fracture  of  the  scapula,  and  3 of  the  4 indicated  “probably  correct”  at  the  0.315-lp/mm 
resolution. 

The  information  content  of  this  drastic  chest  trauma  case  was  found  to  be  too  great 
for  the  best  television  resolution  and  analysis  conditions  employed.  Revisions  of  either  of 
these  factors,  eg,  longer  and  more  detailed  examination,  could  possibly  make  images  at  the 
(best)  resolution  useful.  Clearly,  any  poorer  resolution  would  be  inadequate,  although  since 
group  1 radiologists  were  able  to  diagnose  the  fractured  scapula,  this  would  appear  to  indicate 
that  given  some  additional  information  (such  as  clinical  or  symptomatic  data),  adequate  diag- 
noses could  be  made  even  at  a slightly  lower  resolution. 


CASE  FOUR.  NORMAL  CHEST 

For  the  Diagnoses  Evaluation  Tests,  and  at  the  second  resolution  level.  0.157  Ip/mm, 

3 of  the  4 group  1 radiologists  indicated  “cannot  decide”  evaluations  for  2 or  more  of  the  5 
diagnoses.  The  fourth  radiologist’s  evaluations  were  in  the  “minor  disagreement”  category. 

At  the  third  level,  0.3 1 5 Ip/mm,  only  2 of  the  radiologists  made  a “cannot  decide”  evaluation, 
and  each  was  for  the  pneumothorax  diagnosis.  The  overall  quality  of  the  evaluations  at  this 
resolution,  based  upon  majority  vote  analysis,  was  between  the  “moderate  disagreement”  and 
“minor  disagreement”  categories,  but  somewhat  closer  to  the  latter. 

In  the  Image  Diagnosis  Tests,  one  group  2 radiologist  evaluated  all  resolutions  as 
normal.  The  remaining  3 radiologists  established  diagnoses  which,  at  0. 1 57-lp/mm  resolution, 
related  to  hilar  mass/enlarged  pulmonary  arteries.  They  retained  these  diagnoses  throughout 
the  remainder  of  the  resolutions  and  the  film  reading  with  increasing  confidence  levels.  The 
left  pulmonary  artery  was  enlarged,  but  this  was  neither  u nor  a poshriori.  considered 
a significant  pathological  condition. 

The  results  of  this  case  tend  to  indicate  that  0.3 1 5 Ip/mm  of  resolution  is  satisfactory 
for  establishing  the  normality  of  the  chest  radiograph  vis-a-vis  moderately  advanced  patho- 
logic conditions  other  than  pneumothorax. 


cASi;  MVH,  t'iii;sr  wi ni  acuvh  tuberculosis 


Llf'ective  analysis  of  the  results  tfotn  the  Diagnoses  Lvahiation  Test  for  this  ease  were 
hampered  by  the  fael  that  2 of  the  group  1 radiologists  evaluated  2 and  3 diagnoses  as 
"eannot  deeide”  upon  reading  the  original  radiograph.  In  faet.  Radiologist  I)  evaluated  all 
of  the  diagnoses,  e.xeept  normal,  as  "eannot  deeide”  for  all  resolutions.  At  the  third  level. 
0.315  Ip  inm,  this  radiologist  indieated  that  the  normal  diagnosis  was  "deHnitely  wrong.” 

Ihe  2 radiologists  who  did  provide  deeisive  evaluations  for  the  Diagnoses  Evaluation  Tests 
both  performed  with  "eomplete  agreement”  at  the  fourth  level,  0.()3-lp/mm  resolution,  but 
with  " .'loderate  disagreement” and  "minor  disagreement”  at  the  third  level  (0.315  Ip/mm). 

In  the  Image  Diagnosis  Tests.  2 radiologists  in  group  2 had  loeated  the  true  pathology 
(upper  right-lung  nodules)  at  the  lowest  resolution  (0.070  lp,nmi).  In  one  ease,  the  nodules, 
whieh  lay  under  adjaeent  ribs,  were  defined  a*-  "old  rib  fraetures.”  The  eonfidenees  for 
these  diagnoses  were  "low.”  At  the  seeond  level,  all  radiologists  had  deteeted  the  upper 
right-lung  nodules,  but  no  definitive  diagnoses,  other  than  "rib  fraetures,”  were  given.  At 
the  third  level,  one  radiologist  suggested  "probably  TB”  with  a medium-high  eonfidenee 
level.  .\t  the  highest  level,  3 of  the  4 radiologists  indieated  diagnoses  of  tubereulosis,  or  some 
eciuivalent  variant  thereof.  The  eonfidenees  in  these  diagnoses  were  medium  and  high. 

I'he  results  of  this  ease  indieate  that  the  detection  of  tubereulosis  of  this  nature 
requires  between  0.3 1 5 and  0.(i3  Ip/mm  of  resolution  for  whole  ehest  views.  A system  with 
an  overall  resolution  as  low  as  0. 1 57  Ip/mm  would  be  practical  for  this  ease,  assuming  that  a 
/.oom  eapabilif'  existeil.  Such  zoom  capability  woidd  have  to  allow  for  a linear  magnification 
of  at  least  4 to  I . 


CASE  SIX.  im;lvis  WI  TH  frac  ture  oe'  right  pubic  ramus 

It  was  recognized  u priori  that,  due  to  the  subtle  nature  of  the  symptom  of  this  ease, 
inereaseil  resolution  over  the  ehest  film  eases  would  he  required.  As  such,  the  range  of  resolu- 
tions used  was  0. 1 57  to  1 .0  Ip/mm. 

For  the  Diagnoses  Evaluation  Tests,  and  at  the  seeond  level,  0.315  Ip/mm.  75  percent 
of  the  diagnosis  evaluations  by  group  1 were  "eannot  deeide.”  indicating  the  unaeeeptability 
of  this  resolution  level.  At  the  thinl  level,  0.(>3  Ip/mm,  3 of  the  4 radiologists  still  indieated 
one  "eannot  deeide”  evaluation  among  those  diagnoses  other  than  normal.  Nonetheless,  one 
radiologist  did  provide  a perfect  agreement  with  his  radiograph  reading  results  whieh  were 
very  accurate.  At  the  best  television  resolution  of  !.()  Ip/mm,  2 radiologists'  evaluations  were 
in  the  "minor  disagreement”  category,  while  the  other  2 were  in  the  “moderate”  and  “serious 
disagreement”  categories.  The  majority  vote  analysis  at  the  highest  resolution  level.  1 .0  Ip/mm. 
resulted  in  a categorization  of  “minor  disagreement.” 

Three  of  the  4 group  2 radiologists  in  the  Image  Diagnosis  Test  were  able  to  detect  the 
right  pubic  ramus  fracture  in  the  original  film.  At  the  best  television  resolution,  only  one 
radiologist  mentioned  a pelvic  fracture.  The  eonfidenee  level  on  this  diagnosis  was  only 
medium.  The  remaining  television  image-based  diagtioses  (at  this  resolution)  related  to  the 
lesions,  the  sacroiliac,  and  the  bone  mineralization  pathology. 

The  negative  results  of  the  Image  Diagnosis  Test  lend  to  mitigate  strongly  against  the 
relatively  gooil  results  of  the  Diagnoses  Evaluation  Test  at  highest  resolution.  This  stems  from 
the  faet  that  the  Diagnoses  livaluation  Test  prompted  Ihe  viewer  to  consider  certain  possibili- 
ties. In  the  more  realistic  ease  of  the  Image  Diagnosis  lest,  the  viewer  was  reipiired  to  locate 


I 

the  problem,  assisted  neither  by  diagnostic  cues  nor  by  symptomatic  data.  Thus,  less  than 
1.01  p/mm  of  resolution  in  this  case  would  very  likely  be  unacceptable.  A 1 .0-1  p/mm  resolu- 
tion would  probably  be  only  a marginally  acceptable  one,  although,  again,  higher  resolution 
could  be  attained  with  a zoom  capability  given  that  the  radiologist  knew  where  to  look  for 
abnormal  condition. 

CASH  SEVEN,  SINUS  WITH  SINUSITIS 

Due  to  the  smali  size  of  the  radiograph,  the  test  resolutions  ranged  from  0.315  to  1.2 
Ip/mm.  Its  visual  symptoms  were  a priori  judged  to  be  mostly  related  to  the  gray-level  change 
attributable  to  the  opacitication  associated  with  the  sinusitis  condition.  C'onsistent  with  this 
assumption  is  the  fact  that,  for  the  Diagnoses  Evaluation  Test  and  a resolution  of  0.63  Ip/mm, 
each  group  1 radiologist  evaluated  one  of  the  abnormal  diagnoses  as  “cannot  decide.”  Fur- 
thermore, 3 of  the  4 radiologists  evaluated  the  facial  fracture  diagnosis  as  “probably  correct,” 
both  at  this  resolution  and  at  the  best  television  resolution,  while  evaluating  it  as  wrong  upon 
reading  the  original  radiograph.  At  the  best  television  resolution,  the  quality  of  the  evalua- 
tions for  the  4 radiologists  were  categorized  as  follows;  two  “serious  disagreement,”  one 
“moderate  disagreement”  and  one  “minor  disagreement.” 

In  the  Image  Diagnosis  Tests,  all  group  2 radiologists  were  able  to  recognize  the 
opacification  of  the  left  sinus  area  at  the  lowest  resolution  of  0.3 1 5 Ip/mm.  At  this  resolution, 

2 of  the  4 radiologists  identified  sinusitis,  or  the  roughly  equivalent  diagnosis  of  sinus  mucosal 
thickening,  with  medium-low  confidence  levels.  At  the  second  level,  0.42  Ip/mm,  3 radiolo- 
gists had  presented  one  of  these  preceding  diagnoses.  Also  at  this  resolution  level,  2 radiolo- 
gists presented  the  diagnosis  of  a fracture  with  low  confidence.  At  the  third  level,  0.63 
Ip/mm,  the  radiologists’  evaluations  were  about  the  same  as  at  the  second  level,  except  that 

3 of  the  4 radiologists  evaluated  a fracture  as  a possibility  with  confidences  ranging  from  low 
to  high.  At  the  highest  resolution  of  1.2  Ip/mm,  2 radiologists  still  indicated  a fracture  diag- 
nosis, while  3 diagnosed  either  sinusitis  or  mucosal  thickening. 

While  the  film  showed  no  fracture,  a dark  streak  in  the  medial  wall  of  the  left  orbit 
was  visible.  This  streak  probably  represents  an  air-fluid  differential,  and  was  mistaken  for  a 
fracture  when  reviewing  the  television  images.  When  the  film  itself  was  evaluated,  none  of 
the  8 radiologists  indicated  fracture  diagnoses. 

This  case  again  appears  to  generate  contlicting  results  between  the  Diagnoses  Evalua- 
tion Test  and  the  Image  Diagnosis  Test.  In  an  actual  clinical  situation,  patient  history  and 
physical  symptomatic  condition  information  would  be  available.  With  this  information,  and 
at  the  second  resolution  level,  0.42  Ip/mm,  a radiologist  could  easily  diagnose  whether  or  not 
a fracture  existed  and  could  also  differentiate  between  sinusitis  and  soft  tissue  swelling.  Thus, 
this  resolution  level  would  be  adequate  for  this  type  of  case. 


CASE  EIGHT,  NORMAL  SKULL 

The  tests  with  this  normal  skull  film  were  to  determine  two  things:  first,  could  the 
large  vascular  structure  lapproximately  2..*'  mm  wide  and  over  15  cm  long)  be  discerned  as 
not  being  a fracture;  and  second,  could  the  calcified  pineal  I of  about  5-mm  diameter)  be 
identified. 


2‘) 


Ill  the  Diagnoses  hivaluation  Test,  and  at  the  second  resolution  level.  0.315  Ip/nim. 

3 of  the  4 group  1 radiologists  indicated  a decisive  diagnosis  of  the  fracture.  Two  indicated 
a fracture  existed  and  one  indicated  that  it  was  not  a fracture.  At  the  third  and  fourth 
levels  (0.(i3  Ip/inm  and  1.2  Ip/mni),  all  t!ie  radiologists  had  an  opinion;  they  were  evenly 
split  as  to  whether  or  not  the  visual  symptom  represented  a fracture.  In  viewing  the  actual 
film,  3 of  the  4 radiologists  changed  their  opinions. 

In  the  Image  Diagnosis  Test,  at  the  second  level,  0.315  Ip/mm.  one  group  2 radiolo- 
gist diagnosed  the  prominent  vascular  structure  as  a fracture  with  low  contidence.  .At  the 
third  level,  0.63  lp;mm,  3 of  the  4 radiologists  mentioned  this  visual  symption;  2 recognized 
it  as  a large  vascular  groove  and  one  diagnosed  it  as  a fracture.  All  of  these  diagnoses  were 
with  medium-low  to  medium  confidence.  For  the  best  resolution  of  1 .2  Ip,  mm.  2 of  the 
radiologists  indicated  the  equivalent  of  normal  diagnoses,  while  a third  felt,  with  low  confi- 
dence. that  there  was  a fracture.  The  fourth  radiologist  could  not  decide  between  a fracture 
and  a prominent  vascular  groove. 

From  the  results,  it  is  clear  that  0.3 1 5 Ip/mrn  of  resolution  made  the  pineal  visible 
and  was  suitable  for  displacement  measurement,  so  a resolution  of  twice  this,  or  0.63  Ip/mm, 
is  considered  adequate  for  this  type  of  image. 

( ASF  NINF,  (TIFST  W1 I H PNEUMOTHORAX 

In  the  Diagnoses  Evaluation  Test  at  the  lowest  resolution  level,  0.079  Ip.  mm. 

3 of  the  group  I radiologists  supplied  “cannot  decide”  eva/ualions  of  the  diagnoses.  The 
fourth  radiologist  supplied  evaluations  for  all  4 of  the  non-normal  diagnoses,  but  all  were 
wrong  evaluations.  At  the  second  level.  0. 1 57  Ip/mm,  the  diagnostic  evaluations  provided 
were  generally  either  in  error  or  "cannot  decide.”  At  the  third  level.  0.315  Ip/mm.  all  but  one 
radiologist  had  diagnoses  evaluations  which  were  categorized  as  either  "moderate  disagreement” 
or  "minor  disagreement,”  but  in  quality  from  poor  to  good;  all  4 correctly  diagnosed  the  true 
pathology  of  pneumothorax.  At  the  highest  level  resolution.  0.63  Ip,  mm.  the  radiologists' 
diagnoses  were  categorized  as:  "complete  agreement.”  “minor  disagreement"  and  "moderate 
disagreement.”  The  two  "moderate  disagreement"  ratings  resulted  from  the  radiologists' 
inability  to  decide  on  the  hemothorax  diagnosis. 

In  the  Image  Diagnosis  Tests,  aiul  for  the  first  and  .second  resolution  levels  (0.079  jnd 
0.157  Ip'mm),  no  consistent  pathology  was  diagnosed  by  the  radiologists.  .At  the  third  level, 

0.3  I 5 Ip/mm,  3 of  the  4 radiologists  diagnosed  pneumothorax  with  confidences  ranging  from 
low  to  high.  At  the  best  television  resolution  of  0.(i3  Ip  mm,  all  4 radiologists  provided  the 
correct  diagnosis  of  left  lung  pneiimothor  ix.  I'he  ct)nfidence  levels  in  this  diagnosis  ranged 
from  medium  to  high. 

The  results  of  this  case  indicate  that  a resolution  ol  0 3 I 5 Iji  mm  is  satisfactory  for 
diagnosing  pneumothorax  with  a reasonable  confidence  level. 

( A.SE  TI  N.  .SKUl  I VVll  M FRA(  l URF 

For  this  case,  the  lact  that  the  pineal  was  m)t  c.dcificd  reiluceil  the  choices  ol  di;ig- 
noses  in  the  Diagnoses  Evaluation  Test  to  either  a lr;icture  i>r  normal  1 he  first  use  of  the 
Iracture  diagnosis  was  at  the  second  resolution  level,  0.315  Ip  mm.  .md  w.is  made  by  3 of 
the  4 group  I radinlogisls.  .At  the  third  level.  0.(i3  Ip  mm.  all  4 radiologisis  gave  the  fracture 


diagnosis,  with  2 indicating  “probably  correct”  and  the  other  two  as  “definitely  correct.” 

At  the  fourth  level,  1.2  Ip/mm,  3 of  the  4 radiologists  felt  that  the  fracture  diagnosis  was 
“definitely  correct.” 

In  the  Image  Diagnosis  Test,  the  results  were  basically  the  same  as  for  the  Diagnoses 
Evaluation  Test.  Thus,  at  the  second  level,  0.3 1 5 Ip/mm,  all  group  2 radiologists  diagnosed 
a fracture  with  confidences  from  low  to  medium-high.  At  the  third  and  fourth  levels  (0.63 
Ip/mm  and  1.2  Ip/mm),  3 of  the  4 radiologists  diagnosed  multiple  fractures  with  high  con- 
fidence levels.  The  fourth  radiologist  merely  diagnosed  parietal  skull  fracture. 

The  results  for  this  case  indicate  that  resolution  levels  greater  than  0.315  Ip/mm  (but 
not  more  than  0.63  Ip/mm)  are  necessary  to  define  both  the  fracture  and  its  extent.  The 
lower  resolution  would  be  acceptable  if  a zoom  capability  were  available  to  increase  twofold 
the  image  resolution. 


TV-BASED  IMAGE  RESULTS  BY  DIAGNOSIS 


NORMAL  CHEST  CASE  DIAGNOSES  (CASES  2,  4,  5,  and  9) 

In  general,  when  working  at  the  lowest  resolution  levels,  results  were  better  when  the 
diagnosis  was  “normal”  than  when  it  was  one  of  the  other  four.  At  the  lowest  resolution 
level,  0.157  Ip/mm,  the  evaluations  of  the  normal  fell  one  each  into  these  categories:  “com- 
plete agreement,”  “minor  disagreement,”  “moderate  disagreetnent”  and  “serious  disagree- 
ment.” The  one  “serious  disagreement”  resulted  from  “cannot  decide”  evaluations  and  not 
from  erroneous  diagnoses.  Thus,  the  normal  diagnosis  requires  the  least  resolution. 


HEMOTHORAX  DIAGNOSES  FOR  CHEST  CASES  (CASES  2,  3,  4,  and  9) 

In  general,  the  diagnosis  for  hemothorax  was  accurate  at  the  fourth  resolution  level 
(0.63  Ip/mm),  and  not  quite  as  accurate  at  the  third  resolution  level  (0.3 1 5 Ip/mm).  For 
cases  2 and  3,  one  and  2 radiologists,  respectively,  had  a “cannot  decide”  at  the  third  re.solu- 
tion,  but  at  the  fourth  resolution  level  all  4 radiologists  agreed  (in  direction)  with  their 
readings  of  the  films.  For  case  four,  all  4 radiologists  agreed  in  direction  with  their  readings 
of  the  film  at  both  the  third  and  fourth  resolution  levels.  Only  for  ca.se  nine  were  there  2 
radiologists  who  still  had  “cannot  decides”  at  the  fourth  resolution  level  (the  other  2 
radiologists  agreed  in  direction).  In  case  nine,  there  was  no  hemothorax  pathology  and 
uncertainty  as  to  what  level  of  hemothorax  pathology  to  look  for  could  have  caused  anomal- 
ous results.  The  resolution  required  to  define  a hemothorax  diagnosis  is  likely  to  be  closer 
to  the  0.63  Ip/mm  resolution  than  0.315  Ip/mm. 


PNEUMOTHORAX  DIAGNOSES  FOR  CHEST  CASES  (CASES  2,  4.  and  9) 

At  the  second  resolution  level,  0.157  Ip/mm,  the  Diagnoses  Evaluation  lest  results 
for  the  group  I radiologists  for  pneumothorax  diagnoses  had  7 of  the  I 2 evaluations  (5S 
percent)  of  “cannot  decide.”  One  of  the  remaining  1 2 was  erroneous,  relative  to  the  diag- 
nostic evaluation  based  upon  film  reading.  At  the  third  level,  0.315  Ip  mm,  75  percent 
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ol  1 2)  ol  tlK‘  diagTiostic  evaluations  were  generally  eorreet.  The  3 remaining  evaluations 
were  "eannot  tleeide.”  At  the  highest  level.  0.63  Ip/min,  all  of  the  diagnosis  evaluations 
agreed  almost  exaetly  with  the  lihn  reailing  results. 

Based  upon  these  test  results,  the  resolution  necessary  tor  pneumothorax  diagnosis 
is  generally  in  the  range  of  0.3 1 5 to  0.63  ip.'mm.  The  latter  resolution  would  likely  be 
satisl'aetory  tor  most  eases  of  diagnosis  ol  pneumothorax. 

.M  hLh(  1 X.SIS  DIAGNOSES  R)R  CIlli.ST  ( ASH.S  (CASHS  2.  3,  4.  and  0) 

The  results  by  group  1 with  atelectasis  diagnoses  at  the  first  and  second  levels  (0.070 
and  0. 1 .37  Ip  mm)  were  generally  in  the  '‘major  disagreement”  category.  These  resolutions, 
while  presenting  the  general  visual  ilata  ol  gross  atelectasis,  did  not  provide  sufficient  detail 
to  discern  between  atelectasis  and  other  similar  pathologies.  At  the  thiril  level.  0.31.3  Ip  inm. 
3 of  4 radiologists’  esaluations  were  categorized  as  follows:  "complete  agreetnent.”  ( 2) 
“minor  disagreement."  and  "moderate  disagreement." 

riiese  results  indicate  that  a system  resolution  equivalent  to  0.313  Ip  inm  could  be 
used  lt)r  diagnosing  atelectasis.  Ibis  would  be  particularly  true  when  patient  history,  symp- 
tom data,  and  a close-up  scanning  capability  were  available. 


PNHCMOMA  DIACNOSl  S HOR  ( HHST  ( ASH.S  ((  ASHS  2.  4.  3.  and  6) 

H’or  the  evalu.ition  ot  the  pneumonia  diagnoses  at  the  first  resiilution  level.  0.076 
Ip  mm,  3 radiologists  iiulicateil  "cannot  decide"  evalu.itions  for  three  of  the  four  eases,  as 
shown  in  .Appendix  H . Hi>r  the  third  and  fourth  levels  (0.3 1 3 Ip  mm  and  0.(i3  Ip  mm,  respec- 
tively I,  the  radiologists’  diagnostic  evaluations  were  categorized  as  "in  agreement”  or  “minor 
disagreement” 

H'rom  the  results  of  these  cases,  a system  resolution  of  0.3 1 3 Ip  inm  is  considered 
feasible  for  a pneumonia  di,ignosis  and.  considering  that  pneumonia  is  frequently  well 
localized,  this  system  resolution  may  be  adequate  if  a close-up  scanning  capability  is  available. 

DISCUSSION 


GENERAL  CONSIDERATIONS 

In  the  introductory  section  of  this  report,  it  was  noted  that  the  function  of  the  RMDS 
in  the  transfer  of  radiographic  information  will  generally  be  to  support  intermittent,  emer- 
gency diagnostic  efforts  at  remote  sites.  Hhe  RMDS  is  not  expected  to  be  employed  for 
routine  radiographic  screening  for  early  stages  of  a pathological  condition.  I'hus.  the  resolu- 
tion rec|uirements  for  the  RMDS  equipment  will  not  be  as  stringent  as  other  applications 
might  dictate. 

rite  cases  which  were  used  in  this  study  are  assumed  to  be  typical  of  those  which  might 
be  transferred  by  the  RMDS  equipment.  With  the  exception  of  the  pelvis  of  case  six.  all  cases 
represented  either  normal  or  obviously  abnormal  pathological  conditions.  Hhe  radiologists 
were  able  to  separate  normal  from  abnormal  conditions  with  a high  degree  of  accuracy. 
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Furthermore,  even  though  no  symptomatic  information,  other  than  the  images,  was  available 
to  the  radiologists,  they  were  able  to  provide  a large  percentage  of  reasonably  accurate 
diagnoses. 

The  testing  involved  the  use  of  only  radiologists  in  the  evaluation  of  radiographic 
information.  In  practice,  the  RMDS  is  likely  to  be  used  in  situations  where  the  alternative 
will  be  to  allow  general  practitioners  (GPs),  MDs,  or  hospitalmen  to  evaluate  the  radiographs 
and  to  prescribe  treatment.  Thus,  in  assessing  the  efficiency  of  a certain  degree  of  resolution 
for  the  RMDS,  there  is  a question  of  whether  or  not  the  radiologist  can  perform  better  using 
the  video  image  than  can  a GP  or  hospitalman  using  the  original  radiograph.  Some  light  can 
be  shed  on  this  question  by  research  conducted  by  DrW  Wilson  of  Long  Beach  Memorial 
Hospital.*  His  research  used  a Westinghouse  Slow-Scan  system  to  compare  the  performance 
of  radiologists  using  television  images  against  the  performance  of  GPs  using  original  radio- 
graphs. This  testing  showed  that,  for  cases  with  positive  diagnoses,  the  radiologists  had  an 
accuracy  30  percent  better  than  the  GPs.  Even  greater  improvement  in  accuracy  can  be 
expected  when  the  performance  of  radiologists  is  compared  to  that  of  hospitalmen. 

It  should  be  noted  that  the  tests  reported  upon  in  this  document  were  directed 
toward  the  analysis  of  the  RMDS  radiographic  transmission  resolution  requirements.  Several 
other  RMDS  equipment  parameters  are  also  important  to  the  visualization  process  for  radio- 
graphic  data,  but  were  not  examined.  These  include  noise  and  dynamic  range. 

C AMERA  AND  TRANSMISSION  SYSTEM  NOISE 

The  higher  the  system  noise  levels,  the  more  difficult  will  be  the  visualization  of 
radiographic  data  and,  thus,  the  lower  the  accuracy  of  any  diagnostic  process.  It  is  expected 
that  system  resolution  requirements  might  be  reduced  if  improvements  can  be  achieved  in 
lowering  the  noise  level. 

DYNAMIC  RANGE  OF  CAMERA  AND  DIGITATION  PROCESS 

A substantial  mismatch  exists  between  the  wide  dynamic  range  displayed  in  radio- 
graphs and  the  limited  dynamic  range  in  the  imaging  capabilities  of  standard  television  camera 
system  such  as  were  used  in  the  tests.  In  addition,  limiting  the  transmission  times  in  digital 
systems  requires  limiting  the  number  of  bits  which  are  used  to  represent  each  picture  clement. 
It  is  certain  from  other  reported  research  that  6 to  8 bits  per  picture  element  will  provide 
satisfactory  gray-level  resolution.  Six  bits  per  picture  element  is  expected  to  be  satisfactory 
for  use  in  the  RMDS.  Improvements  in  the  area  of  dynamic  range  can  also  be  expected  to 
improve  system  resolution. 

In  the  tests  reported  upon  herein,  the  radiologists  had  neither  patient  histories  and 
symptom  information  nor  the  option  to  select  the  area  of  the  radiograph  and  the  concomitant 
magnification  to  be  used.  Furthermore,  it  should  be  noted  that  the  testing  was  conducted  by 
one  radiologist  at  a time  under  highly  artificial  conditions.  These  factors  tend  to  reduce  the 
diagnostic  performance  that  might  otherwise  have  been  achieved.  The  operational  conditions 
under  which  the  RMDS  will  be  used  will  allow  the  radiologists,  as  part  of  a team  effort,  to 

* Private  correspondence  with  Dr  W Wilson  concerning  unpublished  research  conducted  by  him  while  at 
the  University  of  Nebraska. 
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scrulini/.c  the  RMDS-traiist'erred  image,  weigti  it  against  patient  history  and  the  clinical 
observations  of  tltc  on-site  medical  personnel,  and  to  recpiest  other  radiographic  views  or 
additional  test  data  needed  to  reach  a conclusive  diagnosis.  Thus,  while  the  testing  reported 
herein  may  superficially  indicate  a lower  performance  level  than  desired  at  the  resolutions 
used,  the  test  conditions  must  be  weighed  in  any  final  conclusions. 


COMMENT  ON  FILM-READING  RESULTS 

In  general,  the  results  pertaining  to  film-reading  indicate  that  one  cannot  expect 
radiologic  film  interpretation  by  individual  radiologists,  in  isolation,  to  lead  to  an  unequivo- 
cally specific  diagnosis.  The  exception  to  this  arises  in  those  cases  of  obvious  symptomatic 
conditions.  In  part,  this  result  stems  from  the  fact  that  the  diagnoses  provided  forevalua- 
tion did  not  indicate  the  degree  of  the  pathological  condition.  Thus,  a conservative  radiolo- 
gist might  hesitiate  in  unequivocally  ruling  out  a pathological  condition  merely  because  the 
film  did  not  present  an  obvious  case  of  that  pathology. 

In  many  eases,  the  radiographic  presentation  was  such  that  more  than  one  pathology 
might  be  plausible.  I'inal  determination  of  the  diagnosis  would  then  require  patient-history 
information.  In  some  cases,  the  film  reading  results  were  simply  erroneous.  I he  erroneous 
results  can  be  attributed  to  either  inexperience  of  the  radiologists,  or.  in  some  cases,  to  hasty 
analysis  of  the  film. 

A significant  conclusion  related  to  the  film-reading  results  is  that,  due  to  the  ambigu- 
ities of  these  results,  no  single  set  of  <;  priori  diagnoses  can  be  used  as  a basis  for  evaluating  the 
television-based  test  results  for  different  radiologists.  Instead,  the  television-based  test  results 
for  each  radiologist  can  be  compared  only  reasonably  to  the  radiographic  film  reading  results 
for  that  individual  radiologist. 


RESOLUTION  REQUIREMENTS 

An  examination  of  table  7 which  summarizes  the  results  of  the  previous  section  by- 
case  and,  by  diagnosis,  confirms  assumptions  made  prior  to  the  testing;  different  anatomy, 
pathology,  and  traumas  require  different  resolutions  for  diagnosis.  A somewhat  surprising 
conclusion  from  these  results  given  in  table  7 is  that  only  relatively  low  values  of  resolution 
are  required  to  diagnose  many  cases. 

Figure  4 shows  a summary  of  the  data  of  table  7.  In  this  figure,  the  plotted  points 
represent  the  cumulative  sum  of  the  percent  of  test  cases  requiring  a resolution  equal  to  or 
less  than  that  shown  on  the  abscissa.  /\  curve  is  plotted  between  these  points.  It  should  be 
noted  that  the  points  plotted  represent  the  results  from  15  discrete  test  data  points.  The 
fitting  of  a curve  to  these  points  should  not  be  construed  to  mean  that  extrapolation  along 
this  curve  is  completely  accurate.  Nevertheless,  this  curve  does  illustrate  a salient  point.  I'he 
point  is  that  a continuum  of  rc.solutions  exists  which  relate  to  Ihe  fraction  of  ca.ses  that  can 
be  effectively  diagnosed  at  that  resolution.  The  distribution  and  types  of  cases  selected  for 
this  study  were  inteiuleil  to  approximate  the  distribution  and  types  of  cases  which  could  be 
expected  in  an  implementation  of  the  RMDS.  I'hus.  it  is  presumed  here  that  the  data  of 
figure  4 present  at  least  a rough!  approximation  of  the  relative  utility  of  the  various  resolu- 
tions for  an  operational  RMDS. 


p 


TABLH  7.  SUMMARY  OF  RFSOLUTION  RtQUlRHMENTS  DATA. 


Case 

Diagnosis 

Minimum 

Acceptable  Resolution 

Comments 

1 

Fracture 

0.157  Ip/mm 

2 

Chest/atelectasis 

0.315  Ip/mm 

3 

Chest /massive 
injuries 

>0.63  Ip/mm 

4 

Chest/normal 

0.315  Ip/mm 

For  moderately  advanced  pathology 

5 

Chest/active  TB 

0.315-0.63 

Ip/mm 

0.157  Ip/mm  would  be  adequate  with 

4:1  magnification 

6 

Pelvis/fracture  of 
right  pubic  ranius 

>1  Ip/mm 

Magnification  nut  an  adequate  routine 
substitute 

7 

Sinus/sinusitis 

0.42  Ip/mm 

8 

Skull/nonnal 

0.63  Ip/mm 

For  obvious  fracture  conditions 

9 

Chest/pneumothorax 

0.315  Ip/mm 

10 

Skull/fracture 

i' 

Chests/normal 

diagnoses 

0.315-0.63 

Ip/mm 

For  obvious  fracture  conditions 

2,4,5 

>0.157  Ip/mm 

For  obvious  pathology 

2,3,4. 9 

Chests/hemothorax 

diagnoses 

0.63  Ip/mm 

2,4,9 

Chests/pneumo- 
thorax diagnoses 

0.315-0.63 

Ip/mm 

2,  3, 4. 9 

- Chests/atelectasis 
diagnoses 

0.315  Ip/mm 

2,4,  5.9 

Chests/pneumonia 

diagnoses 

>0.315  Ip/mm 

1 


1 


4 


I ^ 


35 


100 


80 


60 

CUMULATIVE 
%OF  TEST 
DATA  CASES 

40 


20 


.1  .2  .3  .4  .5  .6  .7  .8  .9  1.0 

MINIMUM  ACCEPTABLE  RESOLUTION 
AT  50%  CONTRAST  LEVEL 

l igure  4.  ('iiimilative  fraction  of  cases  for  which  resoliilion  was  acceptable. 

Interpretation  of  tiie  data  ol  figure  4 reijiiires  tliat  certain  considerations  germane  to 
acquiring  tliese  data  lie  e.xamined.  A significant  consideration  relative  to  the  data  presented 
is  in  the  meaning  of  the  resolution  numbers.  An  earlier  section  of  this  report  discussed  the 
recording  procedures.  It  was  stated  there  that  a bar  pattern  was  used  to  set  the  system  reso- 
lution. This  was  accomplished  by  defocusing  the  camera  lens  until  tlte  bars  for  the  desired 
spatial  frequency  were  at  about  half  their  highest  contrast.  It  should  be  noted  at  this  point 
that  resolution  cannot  be  considered  a bandpass  function,  with  unit  response  to  the  cut-off 
resolution  and  zero  response  thereafter,  as  shown  in  figure  .“ia.  Rather,  for  a defocused  lens, 
the  actual  system  response  would  resemble  the  curve  shown  in  figure  .Sb.  Thus,  the  test 
recordings  were  made  with  a certain  spatial  frequency,  u)^..  at  a reduced  contrast,  r.  For 
frequencies  above  the  system  still  passed  information,  albeit  at  reduced  contrast.  The 
resolution  numbers  usetl  in  the  foregoing  portions  of  this  report  repre.sent  appro.ximately 
the  .x()-percent  contrast  level.  Since  most  system  resolutions  are  specified  at  5-to-lO  percent 
levels,  the  test  results  herein  presented  must  be  adjusted  to  increased  resolution  values  to  be 
compatible  with  the  normaliy  given  resolution  numbers. 

Another  consideration,  relative  to  the  resolution  data  presented,  is  that  the  given 
resolution  numbers  represent  those  for  imaging  the  full  field  of  the  radiographs  used.  In 
some  cases  (eg,  chest  films  with  tuberculosis),  a somewhat  lower  resolution  than  the  minimum 
acceptable  values  pr.-sented  could  be  used.  This  would  be  possible  if  the  system  had  a zoom 
capability  and  could  image  a smaller  portion  of  the  radiograph  at  increased  image  resolution 
but  with  the  same  overall  system  resolution.  Thus,  a system  with  a resolution  of  0..^  Ip/mm 
over  a .40-by-.4()  cm  field  could  be  ilesigned  to  provide  a ().‘*  Ip/mm  resolution  when  zoomed 
to  a lO-by-IO  cm  fiehl.  The  system  resolution  constraint  then  would  be  the  maximum 
resolution  required  for  the  largest  fiekl  to  be  viewed.  Hie  feasibility  of  this  approach  largely 
rests  on  consideration  of  cases  such  as  number  (i,  the  pelvis  with  a fracture  of  the  right  pubic 
ramus.  In  this  case  it  was  determined  that  a resolution  of  over  I Ip,  mm  was  required.  In 
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Figure  5.  Ideal  versus  actual  response  curves  for  defocused  lens. 


order  to  effectively  use  a zoom  capability,  one  needs  to  know  the  area  on  which  to  concen- 
trate. In  case  six,  even  at  the  best  resolution,  the  Image  Diagnosis  Test  showed  a marginal 
capability  to  define  the  diagnosis.  (Note  that  this  was  an  unprompted  diagnostic  situation.) 

In  the  Diagnoses  Evaluation  Test,  the  best  television  resolution  ( 1.0  Ip/mm)  proved  adequate, 
but  the  radiologists  had  been  prompted  as  to  where  to  look.  Thus,  the  concept  of  decreasing 
the  system  resolution  and  relying  upon  a zoom  capability  depends  upon  having  sufficient 
information  to  know  upon  what  area  to  concentrate.  This  information  can  come  either  from 
viewing  the  whole  film  at  reduced  resolution  (at  the  film)  or  from  prompting  based  upon 
knowledge  of  patient  history  and  symptoms. 

As  stated  in  previous  portions  of  this  report,  the  radiologists  were  provided  with  no 
patient  information  other  than  that  of  the  test  image.  This  is  undoubtedly  an  abnormal  con- 
dition for  evaluation  of  radiographic  images.  Based  upon  the  several  cases  examined  in  this 
study,  appropriate  patient  histories  and  symptom  information  would  very  likely  have  allowed 
the  same  accuracy  of  diagnosis  at  resolutions  lower  than  those  quoted  in  table  7.  Further,  if 
the  test  images  were  concurrently  being  evaluated  by  more  than  one  radiologist,  providing  a 
joint  diagnosis,  it  is  anticipated  that  the  resolution  requirements  for  diagnosis  would  be  less 
stringent  than  as  indicated  in  table  7. 

A further  consideration  relative  to  the  results  of  these  tests  is  that  the  participating 
radiologists  were  introduced  to  the  tests  rather  abruptly  and  with  no  prior  preparation.  Thus, 
the  radiologists,  as  far  as  is  known,  did  not  know  about  the  nature  of  the  tests  until  asked  to 
participate.  They  had  never  tried  to  diagnose  from  television  images  (other  than  using  fiuor- 
oscopy,  which  is  an  entirely  different  situation).  It  seems  reasonable  to  expect  that,  with 
some  preparation  for,  or  training  in  diagnosing  from  television  images  the  participating 
radiologists  could  have  performed  much  better  in  the  tests.  This  could  have  resulted  in 
reduced  resolution  requirements  over  those  presented  in  table  (i. 

Another  consideration  is  that  the  diagnoses  presented  forevaluation  in  the  Diagnoses 
Evaluation  Test  may,  in  some  cases,  have  been  ambiguous.  For  example,  when  asked  to 
evaluate  a fracture  diagnosis,  even  with  the  best  resolution,  the  radiologist  might  not  feel 
certain  that  he  had  not  overlooked  some  extremely  fine  fracture  line.  Similar  ambiguities 
existed  in  evaluation  of  pneumonia  and  tuberculosis  in  the  chest-film  cases. 
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In  holli  the  Diagnoses  hvakiation  l est  and  the  Image  Diagnosis  Test,  it  would  have 
been  desirable  to  have  more  information  about  why  a radiologist  made  the  ehoiees  he  did, 

« and  upon  what  visual  symptoms  or  assumptions  these  were  based. 

OVER.ALL  ACCURACY  OF  PERFORMANCE 

A significant  consideration  in  defining  the  performance  of  radiologists  in  analyzing 
radiographic  information  is  their  overall  accuracy  in  terms  of  normal-abnormal  evaluation. 
Each  radiographic  ease  in  this  test  can  be  unambiguously  classified  as  normal  or  abnormal. 

It  is  useful  to  examine  the  frequency  with  which  the  radiologists  defined  normal  radiographs 
as  abnormal  (false-positive)  and  abnormal  radiographs  as  normal  (false-negative). 

Table  8 summarizes  the  false-positive  and  false-negative  results  for  each  of  the  test 
cases.  These  results  are  broken  down  by  film  and  video  image  results  and  by  radiologist 
group  ( 1 and  2).  Thus,  for  the  film-reading  results  and  each  case,  the  number  of  radiologists 
evaluating  that  case  as  normal,  or  abnormal,  is  listed  in  the  appropriate  column.  Similar 
information  is  provided  on  the  video-image  results.  In  addition,  for  the  video-image  results, 
the  resolution  at  which  the  consensus  shown  was  reached  is  given. 

A significant  result  from  these  data  is  that,  with  the  exception  of  some  false-positive 
readings  for  case  eight,  the  radiologists  were  always  able  to  separate  normal  and  abnormal 
conditions  at  one  of  the  video-image  resolutions  used.  Further,  there  were  no  overall  false- 
negative readings,  and  in  all  the  cases,  except  case  eight,  the  separation  of  normal  from 
abnormal  cases  occurred  at  less  than  the  best  video  image  resolution  used. 

SUMMARY  AND  CONCLUSIONS 


SUMMARY 

Preliminary  tests  were  conducted  using  a video  recording  and  television  system  to 
determine  a quantitative  value  for  the  resolution  required  for  diagnosis  of  radiographic 
images.  The  tests  were  of  2 types:  ( I ) the  radiologists  were  asked  to  evaluate  diagnoses  pre- 
sented to  them,  based  upon  what  they  saw  in  the  test  image  (Diagnoses  Evaluation  Test); 
ami,  (2)  the  radiologists  provided  unprompted  diagnoses,  based  upon  what  they  saw  in  the 
test  images  (Image  Diagnosis  Tests). 

Eaght  radiologists  participated  in  the  testing  and  reviewed  50  images  representing 
10  test  cases.  For  each  test  case,  4 television  images  of  differing  resolution  and  the  original 
radiograph  were  evaluated. 

The  Diagnoses  Evaluation  Test  results  were  evaluated,  using  a qualitative  rating  system 
for  comparing  the  responses  of  the  radiologists  for  the  television  images  to  their  responses 
when  viewing  the  original  film.  A majority-vote  analysis  technique  was  used  to  assess  what 
performance  might  he  achieved  with  radiologists  consulting  with  each  other  on  their  film 
readings. 

13y  considering  the  detailed  and  analyzed  results  of  the  Diagnoses  E.valuation  Test 
and  weighing  these  results  against  the  Image  Diagnosis  Test  results,  conclusions  were  reached 
on  the  minimum  resolution  reriuirements  for  each  test  case.  Additionally,  for  the  5 diagnoses 
common  to  the  5 chest  cases,  ;m  analysis  ol  diagnostic  performance  was  conducted.  Ehis 
analysis  yiekled  ilata  on  minimum  resolution  requirements  baseil  upon  diagnosis  type. 
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TABLE  8.  OVERALL  FALSE-POSITIVE  AND  FALSE-NEGATIVE  RESULTS. 


One  radiologist  in  this  group  evaluated  a "Cannot  Decide 


TtST  PROTOCOL  C ONC  LLISIONS 


The  Jes'  as  tlesigiicil  aiul  concliicleil  contained  some  shortcomings.  Some  of  these 
shortcomings  were: 

(a)  Some  of  the  diagnoses  presented  for  evaluation  were  ambiguous  in  that  they 
didn’t  specify  tlie  degree  of  the  pathology  for  which  to  search; 

(h)  No  interactions  with  the  individually  tested  radiologists  were  used  to  determine 
the  basis  for  their  diagnoses  or  evaluations; 

(c)  The  radiologists  were  not  able  to  review  the  images  in  consultation  with  each 
other; 

(d)  No  preliminary  preparations  were  conducted  to  optimize  the  performance  of 
the  radiologists  in  the  tests;  and 

(e)  I'he  laek  of  any  patient  history  of  symptomatic  data  allowed  ambiguous 
results  in  the  film-reading  analyses. 

In  any  future  tests  of  ihis  nature,  it  is  recommended  that  the  above  factors  be  cor- 
rected. ruture  tests  should  he  multiphased.  In  the  first  phase,  the  radiologist  should  be 
familiarized  with,  or  trained  in,  reading  degiaded  televised  radiographs.  A second  phase 
should  involve  a comparison  of  individual  and  teamed  radiologists,  evaluating  various  resolu- 
tion images  with  no  patient  history  data  la  the  final  phase,  lest  cases  would  be  presented 
with  patient  liistory  and  symptoms.  It  is  further  recommeaded  that,  it  possible,  the  tests  be 
condir  ted  with  a digital  system  to  assess  the  effects  of  image  quantization  on  diagnosis. 


RLSOLUTION  CONCLUSIONS 

I'he  resolution  results  presented  here  are  based  fully  upon  very  limited  tests  and 
resulting  test  data.  As  such,  any  adaptation  of  these  results  should  be  done  with  circum- 
spection. 

An  adequate  nominal  HMDS  system  level  resolution  for  transmitting  radiographic- 
data  for  diagnosis,  when  supplemented  with  patient  history  and  symptom  information,  is 
0.5  line  pair  per  millimeter  at  the  film  under  the  following  conditions; 

(a)  A .50-percent  contrast  level  of  the  system  modulation  transfer  function;  and 

(b)  An  imaged  Held  size  of  about  .^0  by  40  cm.  This  size  is  adequate  for  viewing 

a chest  from  the  top  of  the  clavicle  to  the  costiophenic  angles  and  is  more  than 
needed  for  the  width  of  the  chest. 

This  resolution  is  equivalent  to  approximately  .^00  television  lines  per  picture  height  at 
.50-percent  contrast. 
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APPENDIX  A 


PHOTOGRAPHS  OF  THE  RADIOGRAPHS 
USED  IN  THE  TESTS 


This  appendix  contains  photographs  of  the  10  radiographs  used  in  the  tests  of  this 
report.  The  photographs  show  the  radiographs  which  were  presented  to  the  radiologists, 
except  that  patient  information  has  been  omitted  for  this  publication. 


i 
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l igiiio  Al.  Case  one,  lioiie  Iraetiire. 


I'igure  i\2.  Case  Iwo.eliesI  with  collapsed  lower 
left  lobe. 


figure  \!  (.ise  tliiee,  chest  with  multiple  iiinities 


I igiiie  \4  ( ase  lour,  normal  lemale  chest 


■ 96 

(■'igurc  A5.  Case  five,  chest  with  active  tuberculosis 


I'igure  A7.  Case  seven,  sinus  witli  sinusitis 
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APPENDIX  B 


PHOTOGRAPHS  OF  THE  RECORDED 
TELEVISION  IMAGES  USED  IN  THE  TESTS 


This  appendix  contains  illustrations  of  the  television  images  used  in  the  tests  of  the  report. 
Tlie  images  in  this  appendix  are  presented  in  the  order  they  were  shown  to  the  radiologist.  For  each 
subject,  the  resolution  progresses  from  poorest  (a)  to  best  (d).  These  photographs  were  made  from 
Polaroid  prints  of  a television  monitor.  As  such,  they  contribute  significant  degradation  beyond  the 
degraded  television  images. 

The  resolution  values  quoted  in  the  figure  captions  are  those  measured  at  the  radiograph 

surface. 
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Figure  B2.  Case  two,  chest  with  collapsed  left  lower  lobe  (resolution;  a,  0.079  Ip/mm,  43  tv  lines;  b,  0.1 57  Ip/mni,  85  tv  lines;  c,  0.315 
Ip/mni,  171  tv  lines;  d,  0.63  Ip/mm,  342  tv  lines). 


Figure  B3.  Case  three,  chest  with  multiple  injuries  (re.solution:  a,  0.07^  Ip/mm,  50  tv  lines;  b,  0.157  Ip/inin,  ‘>9  tv  lines;  c,  0.3 15  Ip/mm, 
198  tv  lines;  d,  0.63  Ip/mm,  396  tv  lines). 


Figure  B4.  Case  four,  normal  female  chest  (resolution;  a,  0.079  lp/mm,47  tv  lines;  b,  0.1 57  Ip/mm,  94  tv  lines;  c.  0.315  Ip/mm.  189  tv  lines; 
d.  0.63  Ip  mm,  378  tv  lines). 


Figure  B8.  Case  eight,  normal  lateral  skull  (resolution:  a,  0.157  Ip/mm,  70  tv  lines;  b,  0.3 15  Ip/mm,  141  tv  lines;  c,  0.63  Ip/mm,  282  iv  lines; 
d.  1 .2  Ip'mm.  537  tv  lines). 


Fig’irc  B‘>  Case  nine,  chest  with  pneumothorax  (at  expiration ) (resolution:  a.  0.079  Ip  mm.  41  tv  lines:  b,  0.1  .S7  Ip/mm,  82  tv  lines; 
c.  O.J'  1 5 Ip  mm.  I (i4  tv  lines;  d.  O.bi?  Ip  mm.  .^2'’  tv  lines). 


FigureBlO.  Case  ten.  skull  (lateral ) with  multiple  fractures  (resolution:  a,  0.157  Ip/mm,  70  tv  lines;  b,  0.3  15  ip'mm,  141  tv  lines; 
c,  0.63  Ip'mm,  282  tv  lines;  d.  1 .2  Ip/mm.  537  tv  lines). 


APPENDIX  C 


INSTRUCTIONS  TO  PARTICIPANTS 
IN  DIAGNOSES  EVALUATION  TEST  (GROUP  1 


) 


TEST  OF  RESOLUTION  REQUIREMENTS  FOR  RADIOGRAPH  DIAGNOSIS 


BACKGROUND 

The  Naval  Ocean  Systems  Center,  NOSC'  (formerly  the  Naval  l-lectronics  Laboratory 
Center),  Hiomedical  Engineering  Branch,  is  currently  engaged  in  a program  with  the  objective 
of  providing  improved  health  care  to  remotely  located  Navy  operational  sites  having  less  than 
optini'un  medical  staffing.  This  program  envisions  the  development  of  the  Remote  Medical 
Diagnosis  System  (RMDS).  Tlie  RMDS  will  use  hardware  equipment  to  link  remote  sites, 
having  less  than  optimum  medical  staffing,  to  a diagnosis  center  where  a more  complete 
medical  staffing  exists.  This  link  will  allow  for  the  transfer  of  medical  data  from  the  remote 
site  to  the  diagnostic  center  for  evaluation  and  recommendation  of  treatment.  Typical  data 
that  will  be  transferred  back  to  the  diagnostic  center  consist  of  ECCis,  audio  data,  including 
voice  and  heart  lung  sounds,  and  video  images.  These  video  images  will  be  comprised  of  not 
only  patient  images,  but  also  of  radiographic  images. 

It  is  anticipated  that  the  data  sent  from  the  remote  sites  to  the  diagnostic  center  will 
be  analyzed  at  the  diagnostic  center.  The  diagnosis  and  any  resulting  treatment  recommenda- 
tions will  be  transferred  back  to  the  remote  site  via  a voice  circuit  which  will  be  incorporated 
as  part  of  the  Remote  Medical  Diagnosis  System. 

The  Remote  Medical  Diagnosis  System  will  utilize  standard  Navy  communications 
equipment  available  at  the  time  of  its  implementation.  Due  to  the  limitations  of  the  com- 
munications equipment  expected  to  be  available  at  the  time  of  the  RMDS  implementation, 
this  system  will  only  be  able  to  transfer  still  images.  Even  under  these  circumstances,  though, 
the  transfer  of  high-resolution  images,  typical  of  film-radiography,  could  involve  an  inordi- 
nately large  amount  of  time.  In  order  to  help  reduce  the  amount  of  time  required  to  transfer 
a radiograph,  it  will  be  necessary  to  limit  the  resolution  of  the  images  transferred  over  the 
RMDS.  Therefore,  this  test  in  which  you  are  involved  is  an  attempt  to  resolve  the  question 
of  what  is  the  minimum  resolution  required  to  provide  an  image  from  which  an  accurate 
diagnosis  can  be  made  in  a high  percentage  of  the  cases. 


PROCEDURES 

This  test  will  take  approximately  one  hour  to  complete.  During  this  test  you  will  be 
asked  to  view  a series  of  radiographic  images  on  a television  monitor  with  its  input  originating 
from  a video  tape  recorder.  Eacii  image  will  be  displayed  for  approximately  dO  seconds.  For 
each  image  presented,  you  are  asked  to  fill  out  a test  sheet.  Each  test  sheet  contains  a number 
of  diagnoses  relating  to  that  test  image.  Based  upon  what  you  sec  in  the  test  image  on  the 
monitor,  you  arc  asked  to  assess  each  diagnosis  as  to  one  of  the  following  classifications; 

• Definitely  wrong. 

• Probably  wrong. 
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• Cannot  decide, 

• Probably  correct,  and 

• Definitely  correct. 

Assess  each  image  presented  independently  of  the  other  images  and  each  diagnosis  indepen- 
dently of  any  other  diagnosis.  Tlie  illustration  on  the  next  page  shows  an  example  of  a 
filled-out  test  sheet.  Notice  that  more  than  one  diagnosis  can  be  correct.  If  you  have  any 
comments,  reservations,  or  unresolvable  confusion,  make  appropriate  notes  in  “COMMENTS” 
at  the  top  of  the  form.  Please  print  all  comments.  Be  sure  and  evaluate  each  diagnosis  for 
each  image. 

For  each  test  image,  you  are  free  to  adjust  the  contrast  and  brightness  controls  on  the 
monitor  to  achieve  the  image  representation  that  you  desire. 

If  for  any  image  you  have  had  insufficient  time  to  complete  your  test  sheet  or  desire 
to  look  at  the  image  longer,  please  request  that  the  test  monitor  rerun  that  image  for  you.  At 
the  completion  of  the  video  tape  presentation,  you  will  be  asked  to  evaluate  the  original 
radiographs.  For  each  radiograph,  you  will  fill  out  a test  sheet  in  the  same  manner  as  was 
done  for  the  televised  images. 

Prior  to  the  appearance  of  each  test  image  on  the  screen,  a number  will  appear.  Check 
to  ensure  that  the  number  that  appears  is  the  same  as  the  test  image  number  on  the  top  of  the 
sheet  which  you  are  about  to  fill  in.  “X”  the  box  next  to  the  test  image  number  to  indicate 
that  the  image  number  and  test  sheet  numbers  agree.  In  the  case  of  the  radiographs,  numbered 
gummed  labels  will  be  attached  to  the  radiograph. 


CONCLUSION 

This  experiment  will  employ  a very  limited  number  of  radiologists  for  the  test.  As 
such,  the  results  obtained  from  each  radiologist  will  weigh  heavily  upon  the  final  results.  TIrus, 
it  is  important  that  you  do  not  rush  the  test  and  that  the  analysis  of  each  diagnosis  be  made  as 
accurately  as  you  feel  you  are  able  to  make  it.  Evaluate  each  image  presented  independently 
of  other  images  presented  and  each  diagnosis  independently  of  the  other  diagnoses. 

Should  you  have  any  questions  at  this  time  concerning  these  instructions  or  how  the 
experiment  will  be  run,  ask  the  monitor  to  clarify  the  situation  for  you. 

Once  you  feel  you  understand  these  instructions  and  the  experiment  procedure,  please 
turn  to  the  next  sheet  and  fill  in  the  information  requested.  When  you  have  completed  that 
sheet,  indicate  to  the  experiment  monitor  that  you  are  ready  to  begin  the  testing. 
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TEST  IMAGE  # 


12 


IMAGE  SUBJECT:  CHEST 

COMMENTS: 

#1 


Diagnosis; 


ATELECTASIS 


Definitely 

Wrong 

Probably 

Wrong 

Cannot 

Decide 

Probably 

Correct 

Definitely 

Correct 

#2 


Diagnosis; 

FRACTURE  OF  SCAPULA 


Definitely 

Probably 

Cannot 

Probably 

Definitely 

Wrong 

Wrong 

Decide 

Correct 

Correct 

#3 


Diagnosis: 


HEMOTHORAX 


Definitely 

Probably 

Cannot 

Probably 

Definitely 

Wrong 

Wrong 

Decide 

Correct 

Correct 

#4 


Diagnosis 


HEMATOMA 


Definitely 

Probably 

Cannot 

Probably 

Definitely 

Wrong 

Wrong 

Decide 

Correct 

Correct 

#5 


Diagnosis 


RIB  FRACTURE 


Definitely 

Probably 

Cannot 

Probably 

Definitely 

Wrong 

Wrong 

Decide 

Correct 

Correct 

APPENDIX  D 

INSTRUCTIONS  TO  PARTICIPANTS 
IN  IMAGE  DIAGNOSIS  TEST  (GROUP  2) 


TEST  OF  RESOLUTION  REQUIREMENTS  FOR  RADIOGRAPH  DIAGNOSIS 


BACKGROUND 

The  Naval  Ocean  Systems  Center  (formerly  the  Naval  Electronics  Laboratory  Center), 
Biomedical  Engineering  Branch,  is  currently  engaged  in  a program  with  the  objective  of  pro- 
viding improved  health  care  to  remotely  located  Navy  operational  sites  having  less  than 
optimum  medical  staffing.  This  program  envisions  the  development  of  the  Remote  Medical 
Diagnosis  System  (RMDS).  The  RMDS  will  use  hardware  equipment  to  link  remote  sites, 
having  less  than  optimum  medical  staffing,  to  a diagnosis  center  where  a more  optimum 
medical  staffing  exists.  This  link  will  allow  for  the  transfer  of  medical  data  from  the  remote 
site  to  the  diagnostic  center  for  evaluation  and  recommendation  of  treatment.  Typical  data 
that  will  be  transferred  back  to  the  diagnostic  center  consists  of  ECGs,  audio  data,  including 
voice  and  heart  lung  sounds,  and  video  images.  These  video  images  will  be  comprised  of  not 
only  patient  images,  but  also  of  radiographic  images. 

It  is  anticipated  that  the  data  sent  from  the  remote  sites  to  the  diagnostic  center  will 
be  analyzed  at  the  diagnostic  center.  The  diagnosis  and  any  resulting  treatment  recommenda- 
tions will  be  transferred  back  to  the  remote  site  via  a voice  circuit  which  will  be  incorporated 
as  part  of  the  Remote  Medical  Diagnosis  System. 

The  Remote  Medical  Diagnosis  .System  will  utilize  standard  Navy  communications 
equipment  available  at  the  time  of  its  implementation.  Due  to  the  limitations  of  the  com- 
munications equipment  expected  to  be  available  at  the  time  of  the  RMDS  implementation, 
this  system  will  only  be  able  to  transfer  still  images.  Even  under  these  circumstances,  though, 
the  transfer  of  high  resolution  images,  typical  of  film-radiography,  could  involve  an  inordi- 
nately large  amount  of  time.  In  order  to  help  reduce  the  amount  of  time  required  to  transfer 
a radiograph,  it  will  be  necessary  to  limit  the  resolution  of  the  images  transferred  over  the 
RMDS.  Therefore,  this  test  in  which  you  are  involved  is  an  attempt  to  resolve  the  question 
of  what  is  the  minimum  resolution  required  to  provide  an  image  from  which  an  accurate  diag- 
nosis can  be  made  in  a high  percentage  of  the  cases. 


PROCEDURES 

This  test  will  take  appro'  imately  one  hour  to  complete.  During  this  test  you  will  be 
asked  to  view  a series  of  radiographic  images  on  a television  monitor  with  its  input  originating 
from  a video  tape  recorder.  Each  image  will  be  displayed  for  approximately  90  seconds.  For 
each  image  presented,  you  are  asked  to  fill  out  a test  sheet.  Each  test  sheet  contains  space 
for  a number  of  diagnoses  relating  to  that  test  image.  Based  upon  what  you  see  in  the  test 
image  on  the  monitor,  you  are  asked  to  provide  a list  of  potential  diagnoses.  (Please  print  all 
entries.)  At  least  one  diagnosis  should  be  given.  As  many  diagnoses  should  be  provided  as 
seems  justified  by  what  you  see  in  the  image.  Each  diagnosis  you  provide  should  be  classified 
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as  to  your  confidence  in  that  diagnosis  based  upon  what  you  see  in  the  test  image.  Classify 
your  confidence  in  each  diagnosis  by  checking  one  of  the  following  on  the  test  sheet; 

• Low, 

• Medium-Low, 

• Medium, 

• Medium-High,  and 

• High. 

Assess  each  image  presented  independently  of  the  other  images  and  each  diagnosis  indepen- 
dently of  any  other  diagnosis  you  provide.  The  illustration  on  the  next  page  shows  an 
example  of  a filled-out  test  sheet.  If  you  have  any  comments,  reservations,  or  unresolvable 
confusion,  make  appropriate  notes  in  “COMMENTS”  at  the  top  of  the  form.  Please  print 
all  comments.  Be  sure  and  evaluate  your  confidence  in  each  diagnosis  for  each  image 

For  each  test  image,  you  are  free  to  adjust  the  contrast  and  brightness  controls  on  the 
monitor  to  achieve  the  image  representation  that  you  desire. 

If  for  any  image  you  have  had  insufficient  time  to  complete  your  test  sheet  or  desire 
to  look  at  the  image  longer,  please  request  that  the  test  monitor  rerun  that  image  for  you.  At 
the  completion  of  the  video  tape  presentation,  you  will  be  asked  to  evaluate  the  original 
radiographs.  For  each  radiograph,  you  will  fill  out  a test  sheet  in  the  same  manner  as  was 
done  for  the  televised  images. 

Prior  to  the  appearance  of  each  test  image  on  the  screen,  a number  will  appear.  Check 
to  ensure  that  the  number  that  appears  is  the  same  as  the  test  image  number  on  the  top  of  the 
sheet  which  you  are  about  to  fill  in.  “X”  the  box  next  to  the  test  image  number  to  indicate 
that  the  image  number  and  test  sheet  numbers  agree.  In  the  case  of  the  radiographs,  numbered 
gummed  labels  will  be  attached  to  the  radiograph. 


CONCLUSION 

This  experiment  will  employ  a very  limited  number  of  radiologists  for  the  test.  As 
such,  the  results  obtained  from  each  radiologist  will  weight  heavily  upon  the  final  results. 
Thus,  it  is  important  that  you  do  not  rush  the  test  and  that  each  diagnosis  be  made  as  accur- 
ately as  you  feel  you  are  able  to  make  it.  Evaluate  each  image  presented  independently  of 
other  images  presented  and  each  diagnosis  independently  of  the  other  diagnoses. 

Should  you  have  any  questions  at  this  time  concerning  these  instructions  or  how  the 
experiment  will  be  run,  ask  the  monitor  to  clarify  the  situation  for  you. 

Once  you  feel  you  understand  these  instructions  and  the  experiment  procedure, 
please  turn  to  the  next  sheet  and  fill  in  the  information  requested.  When  you  have  completed 
that  sheet,  indicate  to  the  experiment  monitor  that  you  are  ready  to  begin  the  testing. 


Diagnosis:  . 

CONFIDENCE 

n 

LOW 

MEDIUM- 

LOW 

MEDIUM 

MEDIUM- 

HIGH 

HIGH 

1 

1 

#2 

1 

Diagnosis: 

CONFIDENCE 

— 

MEDIUM- 

LOW 

1 

MEDIUM 

MEDIUM- 

HIGH 

HIGH 

1 

1 

1 

#3 


Diagnosis: 

CONFIDENCE 

Hi/ 

LOW 

MEDIUM- 

LOW 

MEDIUM 

MEDIUM- 

HIGH 

HIGH 

v/ 

1 

#4 

Diagnosis: 


CONFIDENCE 


MEDIUM 

MEDIUM- 

LOW 

LOW 

MEDIUM 

HIGH 

HIGH 

APPENDIX  E 


RESOLUTION  VALUES  FOR  TEST  IMAGES 
(LINE  PAIRS/mm  AND  EQUIVALENT  TV  LINES) 


LEVEL 

CASE 

1 

2 

3 

4 

I 

BONE  FRACTURE 

0.157/66 

0.315/132 

0.63/264 

1.0/419 

2 

CHEST  WITH  ATELECTASIS 

0.079/43 

0.157/85 

0.315/171 

0.63/342 

3 

CHEST  WITH  MULTIPLE  INJURIES 

0.079/50 

0.157/99 

0.315/198 

0.63/396 

4 

CHEST  NORMAL 

0.079/47 

0.157/94 

0.315/189 

0.63/348 

5 

CHEST  WITH  ACTIVE  TB 

0.079/47 

0.157/94 

0.315/189 

0.63/378 

6 

PELVIS  WITH  FRACTURE 

0.157/80 

0.315/160 

0.63/321 

1.0/510 

7 

SINUS  WITH  SINUSITIS 

0.315/100 

0.42/134 

0.63/201 

1.2/383 

8 

SKULL  (LATERAL):  NORMAL 

0.157/70 

0.315/141 

0.63/282 

1.2/537 

9 

CHEST  WITH  PNEUMOTHORAX 

0.079/41 

0.157/82 

0.315/164 

0.63/327 

10 

SKULL  (LATERAL):  WITH 

FRACTURE 

0.157/70 

0.315/141 

0.63/282 

1.2/537 

Case  / Diagnosis 

Resolution  Level 

Evaluation  / Radiologist 

1 

2 

3 

4 

Radiograph 

1 Bone  Fracture 

a.  Fracture 

++  ABCD 

++  ABCD 

++  ABCD 

++  ABCD 

++  ABCD 

b.  Normal 

— ABCD 

— ABCD 

— ABCD 

— ABCD 

— ABCD 

2 Chest  with  Atelectasis 

a.  Atelectasis 

+ B 

++  BC 

++BC 

++  ABC 

++ ABC 

0 AD 

+ AD 

+ AD 

+ D 

+ D 

-C 

b.  Hemothorax 

+ C 

0 B 

OB 

+ B 

+ B 

0 ABD 

- ACD 

— A 

— A 

— A 

c.  Normal 

— ABCD 

— ABCD 

~ ABCD 

— ABCD 

— ABCD 

d.  Pneumonia 

+ C 

+ BC 

++BC 

+ ABC 

+ ABC 

0 ABD 

OAD 

+ A 

-D 

-D 

-D 

e.  Pneumothorax 

0 BD 

0 AD 

OD 

OD 

OD 

-AC 

- B 

- AB 

-B 

-B 

— C 

~C 

— AC 

— AC 

3 Chest  with  Multiple  Injuries 

a.  Atelectasis 

+ B 

++B 

++  B 

++B 

++B 

0 D 

OAD 

+ A 

+ A 

+ A 

-A 

~C 

OD 

OD 

OD 

— C 

— C 

— C 

— C 

b.  Fracture  of  Scapula 

0 ABCD 

+ BD 

+ ABD 

++  ABCD 

++  ABCD 

0 AC 

OC 

c.  Hemothorax 

++C 

++  BC 

++BC 

++BC 

++  BC 

+ B 

+ D 

OAD 

+ AD 

+ AD 

0 AD 

-A 

d.  Hematoma 

0 BCD 

0 BCD 

+ B 

+ ABD 

+ ABD 

-A 

-A 

i 

0 ACD 

OC 

OC 

e.  Rib  Fracture(s) 

+ C 

++BC 

++  BC 

++ ABC 

++  ABC 

0 ABD 

+ D 

+ A 

OD 

OD 

0 A 

OD 

tl 
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APPENDIX  F (Continued) 


Resolution  Level 
Evaluation  / Radiologist 


Case  / Diagnosis 


4 Chest:  Normal 


a.  Atelectasis 

b.  Hemothorax 

c.  Normal 

d.  Pneumonia 

e.  Pneumothorax 

5 Chest  With  Active  Tuberculosis 


a.  Active  Tuberculosis 

b.  Bacterial  Pneumonia 


c.  Normal 


OABD 

-C 


+ C 

OABD 

OABD 

-C 

OABD 

-C 


+ C 

OABD 
+ C 

OABD 


+ B 

- ACD 


-D 

— ABC 
-D 

— ABC 


- ABD 
— C 


— ABCD  — ABCD 


d.  Pleural  Plaque 


Radiograph  1 

— ABCD 

1 

i 

! 

1 

— ABCD 

++  BD 

+ AC 

— ABCD 

— ABCD 

+ ABC 

OD 

+ C 

0 BD 

-A 

— ABCD 

0 BD 

APPENDIX  F (Continued) 


Case  / Diagnosis 

1 

Resolution  Level 

Evaluation  / Radiologist 

2 

1 

3 

1 

4 

Radiograph 

c.  Inter-Troehan  Fracture 

0 ACD 

+ B 

+ C 

+ C 

++C 

of  Left  Femoral  Neck 

-B 

OACD 

0 AD 

-AD 

~ ABD 

~ B 

— B 

1 

d.  Normal 

0 ACD 

OCD 

-ACD 

-ACD 

— ABCD 

-B 

-A 

~B 

~B 

— B 

7 Sinus  with  Sinusitis 

a.  Facial  Fracture 

+ C 

+ ACD 

+ ACD 

+ ACD 

-AC 

0 ABD 

OB 

OB 

-B 

— BD 

b.  Normal 

-A 

-A 

~ ABCD 

— ABCD 

~ ABCD 

— BCD 

— BCD 

c.  Sinusitis 

+ AB 

+ B 

+ AB 

+ AB 

++  B 

OCD 

OAC 

OC 

OC 

+ ACD 

-D 

-D 

-D 

d.  Soft  Tissue  Swelling 

+ B 

+ AB 

+ BC 

+ AB 

++  B 

0 ACD 

OC^ 

0 AD 

OCD 

+ A 

-C 

~D 

8 Skull  (Lateral);  Normal 

a.  Fracture 

0 ABCD 

++C 

++  BC 

++C 

++  A 

+ B 

-AD 

+ B 

- BC 

0 D 

1 

-D 

— D 

-A 

— A 

b.  Normal 

0 ABCD 

+ A 

+ AD 

+ AD 

++D 

OD 

~BC 

0 B 

+ C 

- B 

~C 

-B 

— C 

— A 

c.  Pineal  Displacement 

0 ABCD 

+ C 

+ C 

+ c 

0 ABC 

0 AB 

OB 

-ABD 

-D 

-D 

-AD 

APPENDIX  F (Continued) 


Resolution  Level 
Evaluation  / Radiologist 


Case  / Diagnosis 

1 

2 

3 

4 

Radiograph 

9 Chest  With  Pneumothorax 

a.  Atelectasis 

0 ABD 

0 D 

+ B 

++C 

++C 

-C 

-ABC 

OC 

+ B 

+ B 

-D 

-D 

— AD 

— A 

— A 

b.  Hemothorax 

+ BC 

+ BC 

0 Bc: 

OBC 

— ABCD 

0 D 

OD 

- D 

-D 

-A 

-A 

— A 

— A 

c.  Normal 

0 AD 

0 AD 

-D 

- BD 

— ABCD 

— BC 

- B 

~ ABC 

— AC 

~C 

d.  Pneumonia 

+ C 

+ AC 

OC 

-BD 

— ABCD 

0 ABD 

0 BD 

-BD 

— AC 

— A 

e.  Pneumothorax 

0 ABD 

0 BD 

++  AB 

++  ABC 

++  ABCD 

-C 

-AC 

+ CD 

+ D 

10  Skull  (Lateral)  With  Fracture 

a.  Fracture 

0 ABCD 

+ ABC 

++  AB 

++  ABC 

++  ABCD 

0 D 

+ CD 

+ 1) 

b.  Normal 

0 ABCD 

0 D 

-D 

- D 

— ABCD 

-AC 

~ ABC 

— ABC 

- B 

c.  Pineal  Displacement 

0 ABCD 

+ C 

0 ABCD 

+ A 

0 ABCD 

0 ABD 

L— 

0 BCD 

Legend: 


— Definitely  Wrong 
- Probably  Wrong 
0 Cannot  Decide 
+ Probably  Correct 
++  Definitely  Correct 


A — Radiologist  A 
B Radiologist  B 
C Radiologist  C 
D — Radiologist  D 
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APPENDIX  G 

SAMPLE  OF  RADIOLOGIST  COMMENTS 
FOR  DIAGNOSIS  EVALUATION  TEST 


Test 

'mage 

Case 

Resolution 

(Ip/mm) 

COMMENTS  (Verbatim) 

12 

3 

0.63 

“Cannot  see  any  rib  fractures,  if  any  exist.” 

16 

4 

0.63 

“Will  be  very  difficult  to  show  small  pneumothorax,  even  at  best 
resolution  here.” 

19 

5 

0.315 

''Can  say  film  is  abnormal,  but  still  too  gross  to  tell  nature  of 
process.” 

20 

5 

0.63 

“Can  see  abnormality  fairly  well,  but  can’t  decide  between  the 
various  possibilities.  However,  having  a better  resolution  film 
would  not  be  likely  to  answer  the  question,  since  these  diseases 
can  look  like  each  other.” 

24 

6 

1.0 

“Even  at  best  resolution,  cannot  see  internal  bone  detail  well,  so 
cannot  see  small  fracture  lines.” 

28 

7 

1.26 

“Very  difficult  to  see  soft  tissue  changes  even  at  best  resolution.” 

32 

1 

8 

1.26 

1 

“Very  difficult  to  see  small  fracture  lines  in  the  skull.  Would  need 
fairly  obvious  fracture.” 

36 

9 

0.63 

“Still  not  completely  comfortable  with  this  resolution  for  diag- 
nosis of  pneumothorax.” 

38 

10 

0.315 

"Possibly  these  lines  represent  fractures,  but  too  gross  to  do 
anything  except  equivocate.” 

40 

10 

1.2 

“Probably  a fracture,  but  would  need  other  views  for  further 
determination.” 

43 

3 

Radiograph 

“Gross  rib  fractures  on  original  film  which  did  not  transmit.”* 

45 

5 

Radiograph 

“Resolution  not  the  limiting  factor:  even  with  good  resolution, 
it  is  not  possible  to  make  a specific  diagnosis.” 

46 

6 

Radiograph 

“Fat  stripe  simulating  intertrochanteric  fracture  cannot  definitely 
be  differentiated  by  transmitted  image.” 

47 

7 

Radiograph 

“Looked  like  fracture  on  transmission,  but  need  original  films  to 
say  it  is  not.  So,  the  poor  resolution  can  cause  diagnosis  of 
abnormalities  which  are  not  real.” 

49 

9 

Radiograph 

“There  is  also  a pneumomediastinuma  which  did  not  transmit* 
at  all.” 

* Radiologist  means  that  rib  fracture  did  not  show  up  on  TV  images. 
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APPENDIX  H 


CATEGORIZATION  OF  DIAGNOSIS  EVALUATIONS 


Resolution  Level 

Category  / Radiologist 

Case 

1 

2 

3 

4 

2 

Chest  With  Atelectosis* 

ABCDM 

+ C 

-C 

-BC 

— B 

— B 

ADM 

ADM 

ADM 

3 

Chest  With  Massive  .'njuries 

ABCDM 

— B 

- B 

-B 

ACDM 

ACDM 

— ADM 
C 

4 

Normal  Chest 

-C 

-C 

-ACM 

+ M 

ABDM 

ABDM 

— DM 
B 

- ABCDM 

5 

Chest  With  Active 

-C 

-C 

+ C 

+ AC 

Tuberculosis** 

A 

A 

— A 

6 

Pelvis  With  Fracture 

ABCDM 

ABCDM 

+ B 

-ADM 

— ADM 

— B 

C 

C 

7 

Sinus  With  Sinusitis 

~ B 

— B 

— BM 

-B 

AC  DM 

ACDM 

ACD 

— AM 
CD 

8 

Normal  Skull 

ABCDM 

~BC 

-D 

-D 

ADM 

~BC 

— BC 

AM 

AM 

9 

Chest  With  Pneumothorax 

ABCDM 

ABCDM 

+ A 

+ A 

-D 

-D 

— B 

— BC 

CM 

M 

10 

Skull  With  Fracture 

ABCDM 

- ABCM 

+ ABM 

+ ABCM 

D 

-CDM 

-D 

1 * Radiologist  C’s  film  evaluation  for  the  hemothorax  diagnosis  was  “Cannot  Decide,”  and  thus  diagnosis 

I was  not  used  in  these  quality  assessments. 

*•  Radiologists  B and  D were  not  evaluated  due  to  multiple  “Cannot  Decide”  evaluating  for  final  film 
[;  diagnosis. 

[ Note:  Majority  Vote  (M)  can  be  in  two  categories  due  to  ties. 

11  Legend:  Complete  agreement  (+),  Minor  Disagreement  (-).  Moderate  Disagreement  ( — ),  Serious 

Disagreement  ( ). 
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APPENDIX  I 


SAMPLE  OF  RADIOLOGIST  DIAGNOSES 
FOR  IMAGE  DIAGNOSIS  TEST 


Test 

Image 

Case 

Resolution 

(Ip/mm) 

DIAGNOSIS  (Verbatim) 

Confidence 

Level 

1 

1 

0.157 

“Fracture,  transverse” 

High 

2 

1 

0.315 

“Fx,  transv.” 

High 

3 

1 

0.63 

“Fracture,  transverse” 

High 

4 

1 

1.0 

“Fracture,  transverse” 

High 

5 

2 

0.079 

“Left  pleural  effusion” 

Low 

6 

2 

0.157 

“Left  pleural  effusion” 

Medium-low 

7 

2 

0.315 

“Pleural  scarring” 

Medium 

8 

2 

0.63 

“Pleural  scar  base” 

Medium 

9 

3 

0.079 

“Partial  opac.  chest” 

High 

9 

1 

“Calicified  nodule  lung” 

High 

9 

“Pleural  effusion  base” 

Low 

10 

3 

0.157 

“Partial  opacification  lung” 

High 

10 

“Calcified  nodule  lung” 

Medium 

10 

“ pleural  effusion” 

Low 

11 

3 

0.315 

“Partial  opac.  chest” 

High 

11 

“ infrahilar  mass” 

Medium-high 

11 

“Left  pleural  effusion” 

Medium-low 

11 

“Chest  leads” 

Medium-high 

12 

3 

0.63 

“Partial  opac.  Tr)  chest,  possible  loculated 
pleural  eff.” 

High 

12 

“ infrahilar  mass” 

Medium-high 

12 

1 

1 

“ pleural  effusion” 

Mediuin-higli 

12  ' 

1 

1 

“Chest  leads” 

High 

r 

f 


APPENDIX  I (Continued) 


Test 

Image 

Case 

Resolution 

(Ip/mm) 

DIAGNOSIS  (Verbatim) 

Confidence 

Level 

13 

4 

0.079 

“Hyperinflation” 

Medium-higli 

13 

“Centrally  enlarged  pulmonary  arteries” 

Medium-higli 

14 

4 

0.157 

“Hyperinflation” 

Medium-higli 

14 

“Centrally  enlarged  pulmonary  arteries” 

Medium-high 

14 

“ density” 

Low 

15 

4 

0.315 

“Hyperinflation” 

Medium-high 

15 

“Central  enlarged  pulmonary  arteries” 

Medium-high 

15 

“ *^’’8  density” 

Low 

16 

4 

0.63 

“Hyperinflation” 

Medium-higli 

16 

“Central  enlarged  P.A.”  (“Would  like  to 
have  seen  castophrenic  angle  on  ”) 

Medium-higli 

17 

5 

0.079 

“Hyperinflation” 

Medium-high 

17 

“Old  rib  fractures  on  ” 

Low 

18 

5 

0.157 

1 

“Hyperinflation” 

Medium-higli 

18 

“ (rJujXi^  nodular  densities  or  rib 
fractureT'’^ 

Medium 

19 

5 

0.315 

“Hyperinflation” 

Medium-higli 

19 

“Fractures  of  4th  & 5th  ribs  or  overlying 
nodules” 

High 

20 

5 

0.63 

“Hyperinllation” 

Medium-higli 

20 

“Nodular  densities  (2)  rib  fx  vs  nodules” 

lligli 

21 

6 

0.157 

“?  lytic  lesion  ischium” 

Low 

22 

6 

0315 

“?  lytic  lesion  ischium" 

Low 

22 

“?  sclerotic  lesion  ilium” 

Low 

23 

6 

0.63 

“?  lytic  lesion  ischium" 

Low 

* Comments  provided  by  radiologist. 


Test 

Image  Case 


23 

24 

24 

25 

26 
7 

28  ! 

28 

29  8 

30  8 

30 

31  8 

32  8 

33  9 

33 

34  9 

34 

35  9 

35 

36  9 

36 

37  10 

38  10 

39  10 


Resolution 

(Ip/mm) 


APPENDIX  I (Continued) 


DIAGNOSIS  (Verbatim) 


“?  sclerotic  lesion 

“?  lyti 

c 

lesion  ^ 

“?  scle 

:r( 

)tic  lesion 

m 


ischium 


Air  iiuia  level  j max.  sinus 
‘ max.  sinus  mucosal  thickening 

“ max  sinus  mucosal  thickening’ 

‘ max.  sinus  mucosal  thickening 

‘Absence  of  bone  destruction” 

“?  hole  in  frontal  bone” 

‘Hole  in  frontal  bone” 

‘Lucent  area  top  coronal  suture” 
‘Enlarged  vascular  groove” 

“Large  vascular  groove  — normal  skull 
‘ pleural  density  laterally” 

“Borderline  cardiomegaly” 

“Pleural  calcification  bilaterally” 
“Borderline  cardiomegaly” 

“ ^L^  pneumothorax” 

“Borderline  cardiomegaly” 

“ pneumothorax” 

"Borderline  cardiomegaly” 

“Normal” 

“Skull  fx” 

“Skull  fx  multiple” 


Confidence 

Level 


Medium 

Medium-high 


Medium-low 

Medium-low 

Low 

Medium-low 

Medium 

Medium 

Low 

Medium-high 

Medium-low 

High 

Medium-low 

High 

Medium 


gh 


APPENDIX  I (Continued) 


Test 

Image 

Case 

Resolution 

(Ip/mm) 

DIAGNOSIS  (Verbatim) 

Confidence 

Level 

40 

10 

1.2 

“Multiple  skull  fractures” 

High 

41 

1 

Radiograph 

“Transverse  fx  femur” 

High 

42 

2 

Radiograph 

“Pleural  scar  ± or  fluid 

High 

43 

3 

Radiograph 

“ rib  fxs” 

High 

43 

“ pleural  lesion” 

High 

43 

3 

Radiograph 

“ pleural  fluid” 

Medium-high 

43 

“ infrahilar  density  (mass  lesion)” 

Medium 

44 

4 

Radiograph 

“Centrally  enlarged  pulmonary  arteries” 

High 

45 

5 

Radiograph 

“2  nodules  (rXSXC^  ” 

High 

45 

“Hyperinflation” 

Medium-high 

46 

6 

Radiograph 

“Old  fracture  ischium” 

Medium-high 

46 

“Old  fracture  pubis 

Medium 

47 

7 

Radiograph 

“Mucosal  thickening  max.  sinus” 

High 

48 

8 

Radiograph 

“Normal” 

High 

49 

Radiograph 

“Pneumothorax  ” 

High 

50 

Radiograph 

“Multiple  skull  fractures” 

High 

